Information search apparatus and method, and computer readable memory

• US 6,493,705 B1
• Filed: 09/30/1999
• Issued: 12/10/2002
• Est. Priority Date: 09/30/1998
• Status: Expired due to Fees

• Current Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Original Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Inventors: Kobayashi, Yuji, Takata, Tomomi
• Primary Examiner(s): Corrielus, Jean M.
• Assistant Examiner(s): ALAM, SHAHID AL
• Application Number:
  US09/409,056
• Time in Patent Office: 1,167 Days
• Field of Search: 707/1-5, 707/104.1, 707/500, 707/8, 707/10, 707/513
• US Class Current: 1/1
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

A search request input processing unit holds an input query word in a query word holding unit. An associative word unfolding processing unit acquires associative words in association with the query word held in the query word holding unit with reference to an associative word dictionary. An image content search processing unit using associative words makes a keyword search of image information on the basis of the obtained associative words and the query word. A sensory pattern unfolding processing unit obtains the query word and sensory patterns corresponding to associative words of the query word, and searches image information using feature amounts of the obtained sensory patterns. A search result integration processing unit integrates the search results obtained by the image content word search processing unit and sensory pattern search processing unit.

Claims

1. An information search apparatus for managing a plurality of kinds of multimedia information, and searching the managed multimedia information for desired multimedia information, comprising:

a content word holding unit for storing the multimedia information, and content words which verbalize concepts expressed in the multimedia information in correspondence with each other;

an associative word dictionary for storing the content words and associative words which are associated with the content words in correspondence with each other;

input means for inputting a query word;

first search means for acquiring an associative word corresponding to the query word input by said input means from said associative word dictionary, and searching multimedia information on the basis of the acquired associative word;

extraction means for extracting a feature amount corresponding to the query word input by said input means;

second search means for searching multimedia information on the basis of the feature amount extracted by said extraction means; and

integration means for integrating search results obtained by said first and second search means.

2. The apparatus according to claim 1, wherein said input means can also input a search perspective.

3. The apparatus according to claim 2, further comprising:

a concept discrimination dictionary for storing index words and antithetic concepts corresponding to the index words in correspondence with each other, and

wherein said first search means acquires an index word and antithetic concept corresponding to the query word from said concept discrimination dictionary on the basis of the query word and search perspective input by said input means, and acquires an associative word corresponding to the query word from said associative word dictionary on the basis of the acquired index word and antithetic concept.

4. The apparatus according to claim 1, further comprising:

a holding unit for storing associative words and sensory patterns in correspondence with each other, and

wherein said extraction means acquires a sensory pattern corresponding to the associative word, which corresponds to the query word, from said holding unit, and extracts a feature amount of the acquired sensory pattern as the feature amount corresponding to the query word.

5. The apparatus according to claim 1, wherein the multimedia information is image information.

6. The apparatus according to claim 5, wherein the feature amount includes at least one of color scheme information, composition information, and shape information contained in the image information.

7. The apparatus according to claim 1, wherein said integration means integrates the search results obtained by said first and second search means using first matching levels obtained from the search results of said first search means, and second matching levels obtained from the search results of said second search means.

8. An information search method for managing a plurality of kinds of multimedia information, and searching the managed multimedia information for desired multimedia information, comprising:

the storage step of storing on a storage medium a content word holding unit for storing the multimedia information, and content words which verbalize concepts expressed in the multimedia information in correspondence with each other, and an associative word dictionary for storing the content words and associative words which are associated with the content words in correspondence with each other;

the input step of inputting a query word;

the first search step of acquiring an associative word corresponding to the query word input in the input step from said associative word dictionary, and searching multimedia information on the basis of the acquired associative word;

the extraction step of extracting a feature amount corresponding to the query word input in the input step;

the second search step of searching multimedia information on the basis of the feature amount extracted in the extraction step; and

the integration step of integrating search results obtained in the first and second search steps.

9. The method according to claim 8, wherein the input step includes the step of allowing to also input a search perspective.

10. The method according to claim 9, wherein the storage step also includes the step of storing on said storage medium a concept discrimination dictionary for storing index words and antithetic concepts corresponding to the index words in correspondence with each other, and

the first search step includes the step of acquiring an index word and antithetic concept corresponding to the query word from said concept discrimination dictionary on the basis of the query word and search perspective input in the input step, and acquiring an associative word corresponding to the query word from said associative word dictionary on the basis of the acquired index word and antithetic concept.

11. The method according to claim 8, wherein the storage step also includes the step of storing on said storage medium a holding unit for storing associative words and sensory patterns in correspondence with each other, and

the extraction step includes the step of acquiring a sensory pattern corresponding to the associative word, which corresponds to the query word, from said holding unit, and extracting a feature amount of the acquired sensory pattern as the feature amount corresponding to the query word.

12. The method according to claim 8, wherein the multimedia information is image information.

13. The method according to claim 12, wherein the feature amount includes at least one of color scheme information, composition information, and shape information contained in the image information.

14. The method according to claim 8, wherein the integration step includes the step of integrating the search results obtained in the first and second search steps using first matching levels obtained from the search results in the first search step, and second matching levels obtained from the search results in the second search step.

15. A computer readable memory for storing a program code of an information search process for managing a plurality of kinds of multimedia information, and searching the managed multimedia information for desired multimedia information, comprising:

a program code of the storage step of storing on a storage medium a content word holding unit for storing the multimedia information, and content words which verbalize concepts expressed in the multimedia information in correspondence with each other, and an associative word dictionary for storing the content words and associative words which are associated with the content words in correspondence with each other;

a program code of the input step of inputting a query word;

a program code of the first search step of acquiring an associative word corresponding to the query word input in the input step from said associative word dictionary, and searching multimedia information on the basis of the acquired associative word;

a program code of the extraction step of extracting a feature amount corresponding to the query word input in the input step;

a program code of the second search step of searching multimedia information on the basis of the feature amount extracted in the extraction step; and

a program code of the integration step of integrating search results obtained in the first and second search steps.

16. An information search apparatus for managing a plurality of kinds of multimedia information, and searching the managed multimedia information for desired multimedia information, comprising:

a content word holding unit for storing the multimedia information, and content words which verbalize concepts expressed in the multimedia information in correspondence with each other;

an associative word dictionary for storing the content words and associative words which are associated with the content words in correspondence with each other;

input means for inputting a query word;

a concept discrimination dictionary for storing index words corresponding to the query word and search perspectives pertaining to the index words in correspondence with each other;

display means for extracting search perspectives pertaining to an index word corresponding to the query word input by said input means from said concept discrimination dictionary, and displaying the extracted search perspectives;

designation means for designating a desired one of the search perspectives displayed by said display means;

first search means for acquiring an associative word corresponding to the query word input by said input means from said associative word dictionary, and searching multimedia information on the basis of the acquired associative word;

second search means for extracting a feature amount corresponding to the query word input by said input means, and searching multimedia information on the basis of the extracted feature amount; and

integration means for integrating search results obtained by said first and second search means on the basis of the search perspective designated by said designation means.

17. The apparatus according to claim 16, wherein said concept discrimination dictionary also stores antithetic concepts corresponding to the index words, and

wherein said first search means acquires an index word and antithetic concept corresponding to the query word from said concept discrimination dictionary on the basis of the query word and search perspective input by said input means, and acquires an associative word corresponding to the query word from said associative word dictionary on the basis of the acquired index word and antithetic concept.

18. The apparatus according to claim 17, further comprising:

a holding unit for storing associative words and sensory patterns in correspondence with each other, and

wherein said extraction means acquires a sensory pattern corresponding to the associative word, which corresponds to the query word, from said holding unit, and extracts a feature amount of the acquired sensory pattern as the feature amount corresponding to the query word.

19. The apparatus according to claim 17, wherein the multimedia information is image information.

20. The apparatus according to claim 19, wherein the feature amount includes at least one of color scheme information, composition information, and shape information contained in the image information.

21. An information search method for managing a plurality of kinds of multimedia information, and searching the managed multimedia information for desired multimedia information, comprising:

the input step of inputting a query word;

the storage step of storing on a storage medium a content word holding unit for storing the multimedia information, and content words which verbalize concepts expressed in the multimedia information in correspondence with each other, an associative word dictionary for storing the content words and associative words which are associated with the content words in correspondence with each other, and a concept discrimination dictionary for storing index words corresponding to the query word and search perspectives pertaining to the index words in correspondence with each other;

the display step of extracting search perspectives pertaining to an index word corresponding to the query word input in the input step from said concept discrimination dictionary, and displaying the extracted search perspectives;

the designation step of designating a desired one of the search perspectives displayed in the display step;

the first search step of acquiring an associative word corresponding to the query word input in the input step from said associative word dictionary, and searching multimedia information on the basis of the acquired associative word;

the second search step of extracting a feature amount corresponding to the query word input in the input step, and searching multimedia information on the basis of the extracted feature amount; and

the integration step of integrating search results obtained in the first and second search steps on the basis of the search perspective designated in the designation step.

22. The method according to claim 21, wherein said concept discrimination dictionary also stores antithetic concepts corresponding to the index words, and

the first search step includes the step of acquiring an index word and antithetic concept corresponding to the query word from said concept discrimination dictionary on the basis of the query word and search perspective input in the input step, and acquiring an associative word corresponding to the query word from said associative word dictionary on the basis of the acquired index word and antithetic concept.

23. The method according to claim 21, wherein the storage step also includes the step of storing on said storage medium a holding unit for storing associative words and sensory patterns in correspondence with each other, and

the second search step includes the step of acquiring a sensory pattern corresponding to the associative word, which corresponds to the query word, from said holding unit, and extracting a feature amount of the acquired sensory pattern as the feature amount corresponding to the query word.

24. The method according to claim 21, wherein the multimedia information is image information.

25. The method according to claim 24, wherein the feature amount includes at least one of color scheme information, composition information, and shape information contained in the image information.

26. A computer readable memory for storing a program code of an information search process for managing a plurality of kinds of multimedia information, and searching the managed multimedia information for desired multimedia information, comprising:

a program code of the input step of inputting a query word;

a program code of the storage step of storing on a storage medium a content word holding unit for storing the multimedia information, and content words which verbalize concepts expressed in the multimedia information in correspondence with each other, an associative word dictionary for storing the content words and associative words which are associated with the content words in correspondence with each other, and a concept discrimination dictionary for storing index words corresponding to the query word and search perspectives pertaining to the index words in correspondence with each other;

a program code of the display step of extracting search perspectives pertaining to an index word corresponding to the query word input in the input step from said concept discrimination dictionary, and displaying the extracted search perspectives;

a program code of the designation step of designating a desired one of the search perspectives displayed in the display step;

a program code of the first search step of acquiring an associative word corresponding to the query word input in the input step from said associative word dictionary, and searching multimedia information on the basis of the acquired associative word;

a program code of the second search step of extracting a feature amount corresponding to the query word input in the input step, and searching multimedia information on the basis of the extracted feature amount; and

a program code of the integration step of integrating search results obtained in the first and second search steps on the basis of the search perspective designated in the designation step.