SYSTEM AND METHOD OF FINDING DOCUMENTS RELATED TO OTHER DOCUMENTS AND OF FINDING RELATED WORDS IN RESPONSE TO A QUERY TO REFINE A SEARCH

• US 20080016050A1
• Filed: 07/12/2007
• Published: 01/17/2008
• Est. Priority Date: 05/09/2001
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Stensmo, J.
• Granted Patent: US 9,064,005 B2
• Time in Patent Office: Days
• US Class Current: 707/4
• CPC Class Codes: G06F 16/3325 Reformulation based on results of preceding query
• CPC Class Codes: G06F 16/3346 using probabilistic model
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access

Abstract

A computer-implemented system and method is disclosed for retrieving documents using context-dependant probabilistic modeling of words and documents. The present invention uses multiple overlapping vectors to represent each document. Each vector is centered on each of the words in the document and includes the local environment. The vectors are used to build probability models that are used for predictions of related documents and related keywords. The results of the statistical analysis are used for retrieving an indexed document, for extracting features from a document, or for finding a word within a document. The statistical evaluation is also used to evaluate the probability of relation between the key words appearing in the document and building a vocabulary of key words that are generally found together. The results of the analysis are stored in a repository. Searches of the data repository produce a list of related documents and a list of related terms. The user may select from the list of documents and/or from the list of related terms to refine the search and retrieve those documents which meet the search goal of the user with a minimum of extraneous data.

Claims

1. A processor implemented method of finding related documents that are relevant to one or more known documents, comprising:

based on the known documents, automatically finding relevant keywords;

using the keywords, automatically searching for documents that are relevant to the keywords to generate a search result list that includes the known documents; and

removing the known documents from the result list to obtain a list of the related documents.

2. The method of claim 1, further comprising selecting the known documents.

3. The method of claim 2, wherein selecting the known documents comprises checking selection boxes corresponding to the known documents.

4. The method of claim 1, wherein finding the relevant keywords comprises using a predetermined probability model.

5. The method of claim 4, wherein using the probability model comprises calculating Mutual Information and bias coefficients.

6. The method of claim 5, wherein calculating Mutual Information and bias coefficients comprises modeling conditional probability as a sum of the pair-wise mutual information values.

7. The method of claim 1, further comprising retrieving the related documents.

8. A computer program product having program code stored on a computer usable medium, the program code configured to cause a computer to implement a method for finding related documents that are relevant to one or more known documents, comprising:

a first set of program instructions which, based on the known documents, automatically find relevant keywords;

a second set of program instructions that use the keywords to automatically search for documents that are relevant to the keywords, and to generate a search result list that includes the known documents; and

a third set of program instructions that remove the known documents from the result list to obtain a list of the related documents.

9. The computer program product of claim 8, wherein the first set of program instructions use a predetermined probability model.

10. The computer program product of claim 9, wherein the probability model calculates Mutual Information and bias coefficients.

11. The computer program product of claim 10, wherein the Mutual Information and bias coefficients are calculated by modeling conditional probability as a sum of the pair-wise mutual information values.

12. A processor implemented system for finding related documents that are relevant to one or more known documents, comprising:

a first module for automatically finding relevant keywords based on the known documents;

a second module that uses the keywords for automatically searching for documents that are relevant to the keywords, to generate a search result list that includes the known documents; and

a third module for removing the known documents from the result list to obtain a list of the related documents.