System for searching internet using automatic relevance feedback

• US 6,078,917 A
• Filed: 12/18/1997
• Issued: 06/20/2000
• Est. Priority Date: 12/18/1997
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Paulsen, Robert Charles Jr., Martino, Michael John
• Primary Examiner(s): Amsbury, Wayne
• Assistant Examiner(s): Terry, Mark
• Application Number:
  US08/993,115
• Time in Patent Office: 915 Days
• Field of Search: 707/5, 707/6
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3346 using probabilistic model
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

A method of retrieving documents from a document database is disclosed. A set of documents is retrieved according to a first search statement. A signature for a first retrieved document, and preferably other documents by searching for words in the first document and removing common words which occur in a relatively high frequency in a natural language in which the first document is written. The document for which the signature was developed is displayed. Responsive to a user indication that a second search is to be made, deriving a second search statement from the signature of the document.

In the preferred embodiment, a "spectrum" of documents is prepared and presented to the user. The signatures of a plurality of documents from the documents retrieved according to the first search statement by searching for words in the documents and removing common words which occur in a relatively high frequency in a natural language in which the documents are written. The spectrum of documents is selected so that the document signatures differ by at least a predetermined amount. The user may select more than one document to derive further search statement. Responsive to determining that the user has selected a plurality of documents in the spectrum of displayed documents, the second search statement is derived from the signatures of the selected documents.

Claims

1. A method of retrieving computerized documents from a document database in a computer memory comprising the steps of:

retrieving documents according to a first search statement;

developing signatures for a plurality of documents by searching for words in the documents and removing common words which occur in a relatively high frequency in a natural language in which the documents are written;

automatically associating selected ones of the documents with a first document according to a degree of match of the signatures of the selected documents with the signature of the first document;

displaying the first document;

responsive to a user indication that a second search is to be made, automatically selecting a set of words from an aggregate signature based on the signatures of the first and associated documents and the aggregate number of occurrences of the set of words in the first and associated documents; and

constructing a second search statement from the selected set of words.

2. The method as recited in claim 1 further comprising the steps of:

summing the signatures of the first and associated documents to create an aggregate signature;

selecting a set of words from the aggregate signature based on their aggregate number of occurrences in the first and associated documents; and

constructing a search statement from the selected set of words.

3. The method as recited in claim 2 wherein the selected set of words are the words with an aggregate number of occurrences are at least as great as a predetermined number.

4. The method as recited in claim 2 wherein the selected set of words are a predetermined number of words having a highest aggregate number of occurrences.

5. The method as recited in claim 2 further comprising the step of selecting a search statement rule for constructing the second search statement.

6. The method as recited in claim 5 wherein a plurality of search rules are available for selection and arrayed in a narrowest to broadest order, and the method further comprises the steps of:

first selecting a narrow search rule for the creation of the second search statement;

creating the second search statement according to the narrow rule and the set of selected words;

retrieving documents according to the second search statement;

upon an indication from the user that a broader search is desired, selecting a broader rule for the creation of a third search statement; and

creating the third search statement according to the broader rule and the selected set of words.

7. The method as recited in claim 6 further comprising the steps of:

detecting an indication that one of the documents in the spectrum is unacceptable; and

creating the second search statement considering the signature of the unacceptable document.

8. A system including processor, memory and display for retrieving computerized documents from a document database comprising:

means for retrieving documents according to a first search statement;

means for developing signatures for a plurality of documents by searching for words in the documents and removing common words which occur in a relatively high frequency in a natural language in which the documents are written;

means for automatically associating selected ones of the documents with a first document according to a degree of match of the signatures of the selected documents with the signature of the first document;

means for displaying the first document;

means responsive to a user indication that a second search is to be made for automatically selecting a set of words from an aggregate signature based on the signatures of the first and associated documents and the aggregate number of occurrences of the set of words in the first and associated documents; and

means for constructing a second search statement from the selected set of words.

9. The system as recited in claim 8 further comprising:

means for summing the signatures of the first and associated documents to create an aggregate signature;

means for selecting a set of words from the aggregate signature based on their aggregate number of occurrences in the first and associated documents; and

means for constructing a search statement from the selected set of words.

10. The system as recited in claim 9 wherein the selected set of words are the words with an aggregate number of occurrences are at least as great as a predetermined number.

11. The system as recited in claim 9 wherein the selected set of words are a predetermined number of words having a highest aggregate number of occurrences.

12. The system as recited in claim 9 further comprising means for selecting a search statement rule for constructing the second search statement.

13. The system as recited in claim 12 wherein a plurality of search rules are available for selection and arrayed in a narrowest to broadest order, and the system further comprises:

means for first selecting a narrow search rule for the creation of the second search statement;

means for creating the second search statement according to the narrow rule and the set of selected words;

means for retrieving documents according to the second search statement;

means responsive to an indication from the user that a broader search is desired for selecting a broader rule for the creation of a third search statement; and

means for creating the third search statement according to the broader rule and the selected set of words.

14. A computer program product in a computer readable medium for retrieving documents from a document database comprising:

means for retrieving documents according to a first search statement;

means for developing signatures for a plurality of documents by searching for words in the documents and removing common words which occur in a relatively high frequency in a natural language in which the documents are written;

means for associating selected ones of the documents with a first document according to a degree of match of the signatures of the selected documents with the signature of the first document;

means for displaying the first document;

means responsive to a user indication that a second search is to be made for automatically selecting a set of words from an aggregate signature based on the signatures of the first and associated documents and the aggregate number of occurrences of the set of words in the first and associated documents; and

means for constructing a second search statement from the selected set of words.

15. The product as recited in claim 14 further comprising:

means for summing the signatures of the first and associated documents to create an aggregate signature;

means for selecting a set of words from the aggregate signature based on their aggregate number of occurrences in the first and associated documents; and

means for constructing a search statement from the selected set of words.

16. The product as recited in claim 15 wherein the selected set of words are the words with an aggregate number of occurrences are at least as great as a predetermined number.

17. The product as recited in claim 15 wherein the selected set of words are a predetermined number of words having a highest aggregate number of occurrences.

18. The product as recited in claim 15 further comprising means for selecting a search statement rule for constructing the second search statement.