Method of search content enhancement

• US 8,014,997 B2
• Filed: 09/20/2003
• Issued: 09/06/2011
• Est. Priority Date: 09/20/2003
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Tu, Chien-Chiao, Kim, Noon Ju, Huang, Chen-Yu
• Primary Examiner(s): Saint Cyr; Leonard
• Application Number:
  US10/664,449
• Publication Number:
  US 20050065773A1
• Time in Patent Office: 2,908 Days
• Field of Search: 707 3- 5, 704 7- 10
• US Class Current: 704/7
• CPC Class Codes: G06F 16/3338 Query expansion
• CPC Class Codes: G06F 16/38 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 40/242 Dictionaries

Abstract

Whenever a document is going to be included into the textual database, a semantic binder is used to associate the document with one or more semantic nodes which are defined in a semantic taxonomy. When a search is performed, a search application looks through a semantic dictionary (which contains a table mapping queries to nodes on the semantic taxonomy) to see whether any corresponding semantic node can be found for the searchers query. If a match is found, the search application transforms the user'"'"'s query into [“original query” OR “semantic node”] so that relevant documents, even they do not contain any user'"'"'s keyword, can also be found in the database. The system binds semantic nodes arranged in a hierarchical structure of the taxonomy using a Log Analyzer which periodically looks through the system log for new queries and through textual indices for documents added to the database to generate the semantic dictionary and to bind the semantic nodes to the queries in the textual indices of the documents.

Claims

1. A self-enhancing search system for automatically providing expanded keyword searches comprising:

a semantic taxonomy stored on least one computing device, the semantic taxonomy containing semantic node terms in a hierarchical structure, each semantic node term identifying groups of related keywords;

a search system text analyzer stored on the least one computing device, wherein the search system text analyzer periodically looks through a set of documents for a database and identifies keyword terms used in each of the documents that occur in the hierarchical structure;

a semantic binder stored on the least one computing device, the semantic binder for attaching a textual index to each of the documents in the set of the documents, the textual index for each of the documents including at least one semantic node term that identifies keyword terms used in the document;

a semantic dictionary stored on the at least one computing device, the semantic dictionary including a table that maps user query terms to semantic node terms in the semantic taxonomy; and

a relevant document finder stored on the least one computing device, which, only in response to a mapping, based on the semantic dictionary, between a keyword query term in a users keyword query entry with a semantic node term, automatically, without user intervention,

enhances the users keyword query entry with the semantic node term to create an enhanced keyword query, the enhanced keyword query including both the keyword query term and the semantic node term, and,

based on the enhanced keyword query, locates documents in the set of documents that contain a match for the keyword query term and documents in the set of documents that do not contain a match for the keyword query term in the users keyword query but contain other keyword search terms that are linked to the keyword query term by the semantic node term mapped to the keyword query term in the semantic dictionary to thereby increase the number of documents returned to the user.

2. The search system of claim 1 including a semantic dictionary builder which systematically examines the system log off line for new keyword queries to increase the keyword terms in the hierarchical structure and associate them with one or more semantic node terms used to identify the nodes of that structure.

3. The search system of claim 2 including ranking the results of searches using the enhanced queries to place terms in the semantic dictionary in order of most often used keyword query terms to reduce table lookup time.

4. The search system of claim 3, wherein the semantic dictionary builder includes:

a sub-module that identifies domain specific terms in a given keyword query, using domain specific glossary;

a sub-module that finds synonyms and related terms for the identified keyword query terms, using a domain specific thesaurus;

a sub-module that finds that statistically close terms to the identified keyword query terms; and

a sub-module that identifies relevant domain specific categories for the identified keyword terms, using domain specific ontology.

5. The search system of claim 4, wherein the dictionary builder includes:

a sub-module that binds keywords in the identified categories of the hierarchical structures, using the results of the text analyzer.

6. The search system of claim 5, wherein the semantic binder includes:

a sub-module that adds new doc-query links to the meta-data of the corresponding textual index entries to link the documents to the semantic node terms in the hierarchical structure.

7. A computer program comprising program code embodied in at least one non-transitory computer-readable storage medium, which when executed, enables a computer system to implement a method of automatically providing expended keyword searches to increase the scope of keyword searches, the method comprising:

creating a semantic taxonomy containing semantic node terms in a hierarchical structure, each semantic node term identifying groups of related keywords;

periodically looking through a set of documents for a database to identify any keyword terms used in each of the documents that occur in the hierarchical structure;

attaching a textual index for to each of the documents, the textual index for each of the documents including at least one semantic node term that identifies a keyword term used in the document;

creating a table that maps user query terms to semantic node terms in the semantic taxonomy for a semantic dictionary; and

automatically, without user intervention, and only in response to a mapping, based on the semantic dictionary, between a keyword query term in a users keyword query entry with a semantic node term, enhancing the users keyword query to create an enhanced keyword query by adding the semantic node term to the keyword query term, the enhanced keyword query including both the keyword query term and the semantic node term; and

,

based on the enhanced keyword query, locating documents in the set of documents that contain a match for the keyword query term in the users keyword query and documents in the set of documents which do not contain a match for the keyword query term in the users keyword query but contain other keyword search terms that are linked to the keyword query term by the semantic node term mapped to the keyword query term in the semantic dictionary to thereby increase the number of documents returned to the user.

8. The computer program of claim 7, the method further comprising systematically off line examining new user keyword queries in a system log to increase the keyword terms in the semantic dictionary and associates them with one or more semantic node term.

9. The computer program of claim 8, the method further comprising ranking the results of searches using the enhanced queries to place keyword query terms in order of most used keyword terms to reduce table lookup time.

10. The computer system program of claim 9,

identifying domain specific keyword terms in a given query, using domain specific glossary;

finding synonyms and related terms for the identified keyword terms, using domain specific thesaurus;

finding other statistically close keyword terms; and

identifying relevant domain specific categories for that identified keyword terms, using domain specific ontology.

11. The computer program of claim 10, the method further comprising binding keyword queries in the identified semantic taxonomy categories, using the original results of the semantic binder.

12. The computer program of claim 11, the method further comprising adding new doc-query links to the meta-data of the textual index entries to link the documents to the semantic taxonomy categories.

13. A method for a computer search system to interrogate a database that automatically provides expanded keyword search queries comprising:

providing a semantic taxonomy on at least one computing device, the semantic taxonomy containing semantic node terms in a hierarchical structure, each semantic node term identifying groups of related keywords;

providing a search system text analyzer on the at least one computing device, wherein the search system text analyzer periodically looks through a set of documents for a database and identifies keyword terms used in each of the documents that occur in the hierarchical structure;

using a semantic binder stored on the at least one computing device for attaching a textual index to each of the documents in the set of documents, the textual index for each of the documents including at least one semantic node term that identifies keyword terms used in the document;

creating a table that maps user query terms to semantic node terms in the semantic taxonomy for a semantic dictionary; and

providing a relevant document finder on the at least one computing device, which, only in response to a mapping, based on the semantic dictionary, between a keyword query term in a users keyword query entry with a semantic node term, automatically, without user intervention,

enhances the users keyword query entry with the semantic node term to automatically create an enhanced keyword query, the enhanced keyword query including both the keyword query term and the semantic node term, and,

based on the enhanced keyword query, locates documents of the set of documents that contain a match for the keyword query term and documents of the set of documents that do not contain a match for the keyword query term in the users keyword query but contain other different keyword search terms that are linked to the keyword query term by the semantic node term mapped to the keyword query term in the semantic dictionary to thereby increase the number of documents returned to the user.

14. The search method of claim 13, the method further including using a semantic dictionary builder stored on the at least one computing device which systematically examines a system log off line for new keyword queries to increase the keyword terms in the semantic dictionary and associate them with one or more nodes in the hierarchical structure.