Method and apparatus for document filtering using ensemble filters

• US 20040172378A1
• Filed: 11/14/2003
• Published: 09/02/2004
• Est. Priority Date: 11/15/2002
• Status: Active Grant

• Current Assignee: Justsystems Evans Research Incorporated
• Original Assignee: Justsystems Evans Research Incorporated
• Inventors: Evans, David A., Bennett, Jeffrey, Shanahan, James G., Sheftel, Victor M., Montgomery, Jesse
• Granted Patent: US 7,398,269 B2
• Time in Patent Office: Days
• US Class Current: 707/1
• CPC Class Codes: G06F 16/337 Profile generation, learning or modification
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

A technique for representing an information need and employing one or more filters to select documents that satisfy the represented information need, including a technique of creating filters that involves (a) dividing a set of documents into one or more subsets such that each subset can be used as the source of features for creating a filtering profile or used to set or validate the score threshold for the profile and (b) determining whether multiple profiles are required and how to combine them to create an effective filter. Multiple profiles can be incorporated into an individual filter and the individual filters combined to create an ensemble filter. Ensemble filters can then be further combined to create meta filters.

Claims

1. A process for creating a filter for selecting documents, comprising:

identifying a plurality of profiles;

creating a sub-filter for each of said plurality of profiles, each of said sub-filters having an input and an output;

connecting each of said inputs at a single node; and

combining each of said outputs.

2. A process, as in claim 1, further comprising:

mutiplexing said outputs of said sub-filters to create a first filter output.

3. A process, as in claim 2, further comprising:

combining said first filter output with at least one sub-filter output to create at least one second filter output.

4. A process, as in claim 3, further comprising:

aggregating said first filter output and said second filter output using a function.

5. A process, as in claim 4, wherein said function is a linear function.

6. A process, as in claim 4, wherein said function is a weighted function.

7. A process, as in claim 4, wherein said function is a thresholded aggregation function.

8. A process for creating a filter for selecting documents from a stream of documents, comprising:

identifying a plurality of sets of documents from said stream of documents;

identifying a plurality of profiles corresponding to said plurality of sets of documents;

creating a plurality of sub-filters using said plurality of profiles; and

combining a first one of said plurality of sub-filters with a second one of said plurality of sub-filters to create an ensemble filter.

9. A process, as in claim 8, wherein said plurality of sets of documents further comprises:

a set of documents for training;

a first coherent set of documents identified from said training set of documents; and

a remainder set of documents.

10. A process, as in claim 9, wherein said plurality of profiles further comprises:

a first profile corresponding to said first coherent set of documents; and

.

a remainder profile corresponding to said remainder set of documents.

11. A process, as in claim 10, wherein said plurality of sub-profiles further comprises:

a first sub-filter created using said first profile; and

a remainder sub-filter created using said remainder profile.

12. A process for creating a filter for selecting documents from a database of documents, comprising:

identifying a plurality of sets of documents from said database of documents;

identifying a plurality of profiles corresponding to said plurality of sets of documents;

creating a plurality of sub-filters using said plurality of profiles; and

combining a first one of said plurality of sub-filters with a second one of said plurality of sub-filters to create an ensemble filter.

13. A process, as in claim 12, wherein said plurality of sets of documents further comprises:

a set of documents for training;

a first coherent set of documents identified from said training set of documents; and

a remainder set of documents.

14. A process, as in claim 13, wherein said plurality of profiles further comprises:

a first profile corresponding to said first coherent set of documents; and

.

a remainder profile corresponding to said remainder set of documents.

15. A process, as in claim 14, wherein said plurality of sub-profiles further comprises:

a first sub-filter created using said first profile; and

a remainder sub-filter created using said remainder profile.