SYSTEM AND METHOD FOR HIGH PRECISION AND HIGH RECALL RELEVANCY SEARCHING

• US 20100306206A1
• Filed: 05/28/2010
• Published: 12/02/2010
• Est. Priority Date: 05/29/2009
• Status: Active Grant

• Current Assignee: H5, Inc.
• Original Assignee: H5, Inc.
• Inventors: Jade, Teresa Sunshine, Hedin, Bruce, Bauer, Robert S., Hogan, Christopher Michael, Brassil, Daniel Paul
• Granted Patent: US 8,296,309 B2
• Time in Patent Office: Days
• US Class Current: 707/754
• CPC Class Codes: G06F 16/335 Filtering based on additional data, e.g. user or group profiles filtering in web context G06F16/9535, G06F16/9536

Abstract

A method and system for performing high precision and high recall relevancy searching is provided. According to embodiments of the present invention, a relevance rule is generated based on a user model and language from within one or more relevant and non-relevant documents. A query is created based on the relevance rule wherein the query may be applied to a corpus to identify relevant and non-relevant documents. The relevance rule may be iteratively refined in order to increase the accuracy of the query. The resulting query may be used by a litigator during the discovery phase of a litigation to respond to a request for production.

Claims

1. A computer-implemented method for identifying one or more relevant documents, comprising:

generating, by a computer, a filter for identifying a relevant document based on an initial relevance rule related to a set of documents;

applying, by the computer, the filter to the set of documents thereby identifying a subset of relevant documents;

receiving, by the computer from an assessor, the subset of relevant documents comprising an identification of key information;

generating, by the computer, an updated relevance rule based on the key information and the initial relevance rule;

generating, by the computer, a query based on the updated relevance rule for identifying relevant documents within the set of documents; and

outputting, by the computer, the set of documents within which the relevant documents have been identified.

2. The computer-implemented method of claim 1, wherein generating the query for searching the set of documents comprises:

selecting, by the computer, one or more words from a plurality of words included in the updated relevance rule, and

generating, by the computer, a Boolean query based on the one or more selected words.

3. The computer-implemented method of claim 1, wherein applying the filter to the set of documents to identify the subset of relevant documents comprises:

grouping, by the computer, the set of documents into a plurality of subsets, wherein each subset corresponds to a topic, and

identifying, by the computer, the subset of relevant documents, wherein the topic corresponding to the subset of relevant documents matches a relevant word included in the filter.

5. The computer-implemented method of claim 1, wherein the initial relevance rule comprises a list of relevant words.

6. The computer-implemented method of claim 1, further comprising:

applying, by the computer, the query to a second set of documents to identify a set of relevant documents and a set of non-relevant documents; and

generating, by the computer, an accuracy rating reflecting the accuracy with which the query correctly identified the set of relevant documents and the set of non-relevant documents.

4. The computer-implemented method of claim I, wherein generating the updated relevance rule comprises:

identifying, by the computer, a conflict between the key information and the initial relevance rule,

providing, by the computer, the key information and the initial relevance rule to an assessor to resolve the conflict, and

receiving, by the computer, an updated relevance rule wherein the initial relevance rule has been altered to resolve the conflict between the initial relevance rule and the key information.

7. A system for identifying one or more relevant documents, comprising:

a user modeling module configured to;

generate a filter for identifying a relevant document based on an initial relevance rule related to a set of documents,

generate an updated relevance rule based on the key information and the initial relevance rule,

generate a query based on the updated relevance rule for identifying relevant documents within the set of documents,

an assessment module configured to;

apply the filter to the set of documents thereby identifying a subset of relevant documents, and

receive from an assessor, the subset of relevant documents comprising an identification of key information, and

a classification module configured to output the set of documents wherein the relevant documents have been identified.

8. The systems of claim 7, wherein the user modeling module is further configured to:

select one or more words from a plurality of words included in the initial relevance rule, and

generate a Boolean query including the one or more selected words.

9. The system of claim 7, wherein the assessment module is further configured to:

group the set of documents into a plurality of subsets, wherein each subset corresponds to a topic, and

identify the subset of relevant documents, wherein the topic corresponding to the subset of relevant documents matches a relevant word included in the filter.

10. The system of claim 7, wherein the assessment module is further configured to:

identify a conflict between the key information and the initial relevance rule,

provide the key information and the initial relevance rule to an assessor to resolve the conflict, and

receive an updated relevance rule wherein the initial relevance rule has been altered to resolve the conflict between the initial relevance rule and the key information.

11. The system of claim 7, wherein the initial relevance rule comprises a list of relevant words.

12. The system of claim 7, further comprising a measurement module configured to:

apply the query to a second set of documents to identify a set of relevant documents and a set of non-relevant documents; and

receive an accuracy rating reflecting the accuracy with which the query correctly identified the set of relevant documents and the set of non-relevant documents.