Method and apparatus for profile score threshold setting and updating

• US 20030004927A1
• Filed: 06/05/2002
• Published: 01/02/2003
• Est. Priority Date: 11/02/1999
• Status: Active Grant

• Current Assignee: Justsystems Evans Research Incorporated
• Original Assignee: Clairvoyance Corporation
• Inventors: Zhai, Chengxiang
• Granted Patent: US 6,535,876 B2
• Time in Patent Office: Days
• US Class Current: 707/1
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

A novel approach for filtering documents involves the use of delivery ratio threshold setting technique to set an initial profile score threshold and the use of beta-gamma regulation for dynamic threshold updating. A group of documents is scored pursuant to a user profile. The score for each document is indicative of the relevance of the corresponding document to the user profile. The score can be compared with a profile score threshold to decide if the document should be accepted or rejected. According to one aspect of the invention, the initial threshold is set to a score threshold that approximates an expected ratio of acceptable documents calibrated with respect to a set of reference documents. According to another aspect of the invention, the score threshold can be updated based on the accumulated example documents, user'"'"'s relevance judgment, and the user'"'"'s utility function. The accumulated example documents are first scored against a profile and a ranked list of scored documents is obtained. Each position at the ranked list corresponds to a candidate score threshold as well as a utility value computed based on the relevance status of the example documents. From these candidate threshold points, an optimal utility threshold and a zero utility threshold are determined. Using the optimal utility threshold and the zero utility threshold, a new utility threshold is calculated by interpolating between estimates of the optimal utility threshold and the zero utility threshold. This new utility threshold is used for subsequent information retrieval and filtering.

Claims

1. A computer-implemented method for filtering documents, comprising the steps of:

selecting a document profile and an expected document delivery ratio;

scoring a reference set of documents according to said document profile;

determining an assigned score threshold corresponding to said expected document delivery ratio;

determining a utility function by calculating a utility for each of said documents in said reference set;

determining a first utility threshold based on said utility function;

determining a second utility threshold based on said utility function;

interpolating between said first utility threshold and said second utility threshold to determine an updated score threshold; and

filtering incoming documents based on said updated score threshold.

2. A method as in claim 1, wherein said interpolation is linear.

3. A method as in claim 1, wherein said first utility threshold (θ

_opt) is the highest utility over said reference set and said second utility threshold (θ

_zero) is the highest utility below said first utility threshold that has a non-positive utility over said reference set.

4. A method as in claim 3, wherein:

said interpolation is calculated according to the formula;

θ

=α

·

θ

_zero+(1−

α

)·

θ

_opt

5. A method as in claim 4, wherein said parameter is calculated according to the formula:

α

=β

+(1−

β

)·

e<sup>−

γ

·

M</sup>

6. A system for filtering documents, comprising:

a computer coupled to a network wherein said computer receives documents over said network and transmits documents to an individual user over said network, wherein said user selects a document profile and an expected delivery ratio; and

wherein said computer;

scores a reference set of documents according to said document profile;

determines an assigned score threshold corresponding to said expected document delivery ratio;

determines a utility function by calculating a utility for each of said documents in said reference set;

determines a first utility threshold based on said utility function;

determines a second utility threshold based on said utility function;

interpolates between said first utility threshold and said second utility threshold to determine an updated score threshold; and

filters incoming documents based on said updated score threshold.

7. A system as in claim 6, wherein said interpolation is linear.

8. A system as in claim 6, wherein said first utility threshold (θ

_opt) is the highest utility over said reference set and said second utility threshold (θ

_zero) is the highest utility below said first utility threshold that has a non-positive utility over said reference set.

9. A system as in claim 8, wherein:

said interpolation is calculated according to the formula;

θ

=α

·

θ

_zero+(1−

α

)·

θ

_opt

10. A system as in claim 9, wherein said parameter is calculated according to the formula:

α

=β

+(1−

β

)·

e<sup>−

γ

·

M</sup>