Information search apparatus and method, and computer readable memory

• US 6,493,692 B1
• Filed: 09/30/1999
• Issued: 12/10/2002
• Est. Priority Date: 09/30/1998
• Status: Expired due to Fees

• Current Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Original Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Inventors: Kobayashi, Yuji, Takata, Tomomi
• Primary Examiner(s): STARKS, WILBERT L
• Application Number:
  US09/409,049
• Time in Patent Office: 1,167 Days
• Field of Search: 706/46, 707/3, 707/5, 707/10, 707/513, 717/146
• US Class Current: 706/46
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access

Abstract

An apparatus has an image content word holding unit for storing multimedia information, and content words each of which verbalizes a concept expressed in the multimedia information, and an associative word dictionary for storing content words, associative words associated with the content words, and non-associative words which one does not bilaterally associate with the content words in correspondence with each other. The apparatus acquires associative words corresponding to an input query word from the associative word dictionary, and searches multimedia information on the basis of the acquired associative words using a search processing unit. Also, the apparatus acquires non-associative words corresponding to the input query word from the associative word dictionary, and searches multimedia information on the basis of feature amounts corresponding to the acquired non-associative words using the search processing unit. A search result integration processing unit integrates first and second search results obtained by the search processing unit on the basis of the second search results.

Claims

1. An information search apparatus for managing a plurality of kinds of multimedia information, and searching the managed multimedia information for desired multimedia information, comprising:

an image content word holding unit for storing the multimedia information, and content words which verbalize concepts expressed in the multimedia information in correspondence with each other;

an associative word dictionary for storing the content words and non-associative words which do not evoke bilateral association with the content words in correspondence with each other;

input means for inputting a query word;

first search means for acquiring a non-associative word corresponding to the query word input by said input means from said associative word dictionary, and searching multimedia information on the basis of the acquired non-associative word;

second search means for extracting a feature amount corresponding to the query word input by said input means, and searching multimedia information on the basis of the extracted feature amount; and

integration means for integrating first and second search results obtained by said first and second search means on the basis of the first search results.

2. The apparatus according to claim 1, wherein said integration means lowers matching levels of those search results of the second search results, which correspond to the first search results.

3. The apparatus according to claim 1, wherein said integration means deletes those search results of the first search results, which have matching levels smaller than a predetermined value.

4. The apparatus according to claim 1, wherein said associative word dictionary stores associative words which are associated with the non-associative words, and

said first search means searches multimedia information on the basis of the non-associative words, acquires an associative word corresponding to the non-associative word from said associative word dictionary, and searches multimedia information on the basis of the acquired associative words.

5. The apparatus according to claim 1, wherein said input means can also input a search perspective.

6. The apparatus according to claim 5, further comprising:

a concept discrimination dictionary for storing index words and antithetic concepts corresponding to the index words in correspondence with each other, and

wherein said first search means acquires an index word and antithetic concept corresponding to the query word from said concept discrimination dictionary on the basis of the query word and search perspective input by said input means, and acquires a non-associative word corresponding to the query word from said associative word dictionary on the basis of the acquired index word and antithetic concept.

7. The apparatus according to claim 1, further comprising:

a holding unit for storing associative words and sensory patterns in correspondence with each other, and

wherein said second search means acquires a sensory pattern corresponding to the associative word from said holding unit, and extracts a feature amount of the acquired sensory pattern as the feature amount corresponding to the query word.

8. The apparatus according to claim 1, wherein the multimedia information is image information.

9. The apparatus according to claim 8, wherein the feature amount includes at least one of color scheme information, composition information, and shape information contained in the image information.

10. An information search method for managing a plurality of kinds of multimedia information, and searching the managed multimedia information for desired multimedia information, comprising:

the storage step of storing on a storage medium an image content word holding unit for storing the multimedia information, and content words which verbalize concepts expressed in the multimedia information in correspondence with each other, and an associative word dictionary for storing the content words and non-associative words which do not evoke bilateral association with the content words in correspondence with each other;

the input step of inputting a query word;

the first search step of acquiring a non-associative word corresponding to the query word input in the input step from said associative word dictionary, and searching multimedia information on the basis of the acquired non-associative word;

the second search step of extracting a feature amount corresponding to the query word input in the input step, and searching multimedia information on the basis of the extracted feature amount; and

the integration step of integrating first and second search results obtained in the first and second search steps on the basis of the first search results.

11. The method according to claim 10, wherein the integration step includes the step of lowering levels of those search results of the second search results, which correspond to the first search results.

12. The method according to claim 10, wherein the integration step includes the step of deleting those search results of the first search results, which have matching levels smaller than a predetermined value.

13. The method according to claim 10, wherein said associative word dictionary stores associative words which are associated with the non-associative words, and

the first search step includes the step of searching multimedia information on the basis of the non-associative words, acquiring an associative word corresponding to the non-associative word from said associative word dictionary, and searching multimedia information on the basis of the acquired associative words.

14. The method according to claim 10, wherein the input step includes the step of allowing to also input a search perspective.

15. The method according to claim 14, wherein the storage step also includes the step of storing on said storage medium a concept discrimination dictionary for storing index words and antithetic concepts corresponding to the index words in correspondence with each other, and

the first search step includes the step of acquiring an index word and antithetic concept corresponding to the query word from said concept discrimination dictionary on the basis of the query word and search perspective input in the input step, and acquiring a non-associative word corresponding to the query word from said associative word dictionary on the basis of the acquired index word and antithetic concept.

16. The method according to claim 10, wherein the storage step also includes the step of storing on said storage medium a holding unit for storing associative words and sensory patterns in correspondence with each other, and

the second search step includes the step of acquiring a sensory pattern corresponding to the associative word from said holding unit, and extracting a feature amount of the acquired sensory pattern as the feature amount corresponding to the query word.

17. The method according to claim 10, wherein the multimedia information is image information.

18. The method according to claim 17, wherein the feature amount includes at least one of color scheme information, composition information, and shape information contained in the image information.

19. A computer readable memory which stores a program code of an information search process for managing a plurality of kinds of multimedia information, and searching the managed multimedia information for desired multimedia information, comprising:

a program code of the storage step of storing on a storage medium an image content word holding unit for storing the multimedia information, and content words which verbalize concepts expressed in the multimedia information in correspondence with each other, and an associative word dictionary for storing the content words and non-associative words which do not evoke bilateral association with the content words in correspondence with each other;

a program code of the input step of inputting a query word;

a program code of the first search step of acquiring a non-associative word corresponding to the query word input in the input step from said associative word dictionary, and searching multimedia information on the basis of the acquired non-associative word;

a program code of the second search step of extracting a feature amount corresponding to the query word input in the input step, and searching multimedia information on the basis of the extracted feature amount; and

a program code of the integration step of integrating first and second search results obtained in the first and second search steps on the basis of the first search results.

20. An information search apparatus for managing a plurality of kinds of multimedia information, and searching the managed multimedia information for desired multimedia information, comprising:

an image content word holding unit for storing the multimedia information, and content words which verbalize concepts expressed in the multimedia information in correspondence with each other;

an associative word dictionary for storing the content words, associative words associated with the content words, and non-associative words which do not evoke bilateral association with the content words in correspondence with each other;

input means for inputting a query word;

first search means for acquiring an associative word corresponding to the query word input by said input means from said associative word dictionary, and searching multimedia information on the basis of the acquired associative word;

second search means for acquiring a non-associative word corresponding to the query word input by said input means, and searching multimedia information on the basis of the acquired non-associative word; and

integration means for integrating first and second search results obtained by said first and second search means on the basis of the first search results.

21. The apparatus according to claim 20, wherein said integration means lowers matching levels of those search results of the second search results, which correspond to the first search results.

22. The apparatus according to claim 20, wherein said integration means deletes those search results of the first search results, which have matching levels smaller than a predetermined value.

23. The apparatus according to claim 20, wherein said second search means extracts a feature amount corresponding to the non-associative word, and extracts a feature amount which is irrelevant to the query word.

24. The apparatus according to claim 20, wherein said input means can also input a search perspective.

25. The apparatus according to claim 24, further comprising:

a concept discrimination dictionary for storing index words and antithetic concepts corresponding to the index words in correspondence with each other, and

wherein said first search means acquires an index word and antithetic concept corresponding to the query word from said concept discrimination dictionary on the basis of the query word and search perspective input by said input means, and acquires an associative word corresponding to the query word from said associative word dictionary on the basis of the acquired index word and antithetic concept.

26. The apparatus according to claim 20, further comprising:

a holding unit for storing non-associative words and sensory patterns in correspondence with each other, and

wherein said second search means acquires a sensory pattern corresponding to the non-associative word from said holding unit, and extracts a feature amount of the acquired sensory pattern as the feature amount corresponding to the non-associative word.

27. The apparatus according to claim 20, wherein the multimedia information is image information.

28. The apparatus according to claim 27, wherein the feature amount includes at least one of color scheme information, composition information, and shape information contained in the image information.

29. An information search method for managing a plurality of kinds of multimedia information, and searching the managed multimedia information for desired multimedia information, comprising:

the storage step of storing on a storage medium an image content word holding unit for storing the multimedia information, and content words which verbalize concepts expressed in the multimedia information in correspondence with each other, and an associative word dictionary for storing the content words, associative words associated with the content words, and non-associative words which do not evoke bilateral association with the content words in correspondence with each other;

the input step of inputting a query word;

the first search step of acquiring an associative word corresponding to the query word input in the input step from said associative word dictionary, and searching multimedia information on the basis of the acquired associative word;

the second search step of acquiring a non-associative word corresponding to the query word input in the input step, and searching multimedia information on the basis of the acquired non-associative word; and

the integration step of integrating first and second search results obtained in the first and second search steps on the basis of the first search results.

30. The method according to claim 29, wherein the integration step includes the step of lowering matching levels of those search results of the second search results, which correspond to the first search results.

31. The method according to claim 29, wherein the integration step includes the step of deleting those search results of the first search results, which have matching levels smaller than a predetermined value.

32. The method according to claim 29, wherein the second search step includes the step of extracting a feature amount corresponding to the non-associative word, and extracting a feature amount which is irrelevant to the query word.

33. The method according to claim 29, wherein the input step includes the step of allowing to also input a search perspective.

34. The method according to claim 33, wherein the storage step also includes the step of storing on said storage medium a concept discrimination dictionary for storing index words and antithetic concepts corresponding to the index words in correspondence with each other, and

the first search step includes the step of acquiring an index word and antithetic concept corresponding to the query word from said concept discrimination dictionary on the basis of the query word and search perspective input in the input step, and acquiring an associative word corresponding to the query word from said associative word dictionary on the basis of the acquired index word and antithetic concept.

35. The method according to claim 29, wherein the storage step also includes the step of storing on said storage medium a holding unit for storing non-associative words and sensory patterns in correspondence with each other, and

the second search step includes the step of acquiring a sensory pattern corresponding to the non-associative word from said holding unit, and extracting a feature amount of the acquired sensory pattern as the feature amount corresponding to the non-associative word.

36. The method according to claim 29, wherein the multimedia information is image information.

37. The method according to claim 36, wherein the feature amount includes at least one of color scheme information, composition information, and shape information contained in the image information.

38. A computer readable memory which stores a program code of an information search process for managing a plurality of kinds of multimedia information, and searching the managed multimedia information for desired multimedia information, comprising:

a program code of the storage step of storing on a storage medium an image content word holding unit for storing the multimedia information, and content words which verbalize concepts expressed in the multimedia information in correspondence with each other, and an associative word dictionary for storing the content words, associative words associated with the content words, and non-associative words which do not evoke bilateral association with the content words in correspondence with each other;

a program code of the input step of inputting a query word;

a program code of the first search step of acquiring an associative word corresponding to the query word input in the input step from said associative word dictionary, and searching multimedia information on the basis of the acquired associative word;

a program code of the second search step of acquiring a non-associative word corresponding to the query word input in the input step, and searching multimedia information on the basis of the acquired non-associative word; and

a program code of the integration step of integrating first and second search results obtained in the first and second search steps on the basis of the first search results.