Incorporating rules and knowledge aging in a Naive Bayesian Classifier

• US 20080154813A1
• Filed: 10/26/2006
• Published: 06/26/2008
• Est. Priority Date: 10/26/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Marshall, Stuart H., Penev, Kamen
• Granted Patent: US 7,672,912 B2
• Time in Patent Office: Days
• US Class Current: 706/14
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: H04L 51/212 using filtering or selective blocking

Abstract

The Naïve Bayes Classifier predicts the classification of a set of data based on the features of that data and a series of counts reflecting the information obtained from prior data sets, with one count per feature per class. An external boost can be applied to the counts generated by the NBC to account for external information. Such a boost is added to the counts generated by the NBC, and the boosted counts are then used by the NBC. A boost can be applied to some or all of the counts and the boost for each count can be applied independently. Likewise, the counts can be periodically aged by multiplying the counts with an aging factor of between 0 and 1 per period. Aging factors can be applied uniformly across all counts, or can be individually applied, enabling some counts to age more than others.

Claims

1. One or more computer-readable media comprising computer-executable instructions directed to steps comprising:

obtaining one or more counts generated by a statistical classifier, the one or more counts being based on historical verified data consumed by the statistical classifier;

selecting one or more boost values reflecting information external to the historical verified data; and

boosting the one or more counts with the one or more boost values, the boosted one or more counts being used by the statistical classifier to classify new data.

2. The computer-readable media of claim 1, wherein the boosting the one or more counts with the one or more boost values comprises adding the one or more boost values to the one or more counts.

3. The computer-readable media of claim 1, wherein the boosting the one or more counts with the one or more boost values comprises multiplying the one or more counts with the one or more boost values.

4. The computer-readable media of claim 1, wherein the one or more boost values are independent of one another and correspond to the one or more counts.

5. The computer-readable media of claim 1, wherein the selecting the one or more boost values comprises providing a user interface for viewing or selecting the one or more boost values.

6. The computer-readable media of claim 5, wherein the user interface for selecting the one or more boost values comprises guidance regarding an expected effect of selected boost values.

7. One or more computer-readable media comprising computer-executable instructions directed to steps comprising:

obtaining one or more counts generated by a statistical classifier, the one or more counts being based on historical verified data, comprising one or more data entries, that is consumed by the statistical classifier;

selecting one or more aging factors reflecting a correlation between the historical verified data'"'"'s age and its relevance to new data; and

aging the one or more counts with the one or more aging factors, the aged one or more counts being used by the statistical classifier to classify new data.

8. The computer-readable media of claim 7 comprising further computer-executable instructions for determining the one or more aging factors, the further computer-executable instructions directed to steps comprising:

selecting an initial set of one or more aging factors;

aging the one or more counts with the initial set of one or more aging factors to form an initial aged set of counts;

determining an initial predictive accuracy of the statistical classifier using the initial aged set of counts;

selecting a subsequent set of one or more aging factors;

aging the one or more counts with the subsequent set of one or more aging factors to form an subsequent aged set of counts;

determining a subsequent predictive accuracy of the statistical classifier using the subsequent aged set of counts;

comparing the initial predictive accuracy to the subsequent predictive accuracy; and

selecting the one or more aging factors based on a highest relative predictive accuracy of the statistical classifier using the aged one or more counts.

9. The computer-readable media of claim 7 comprising further computer-executable instructions directed to steps comprising:

maintaining both the one or more counts and the aged one or more counts.

10. The computer-readable media of claim 7, wherein the one or more aging factors are greater than 1.

11. The computer-readable media of claim 7, wherein the aging the one or more counts is performed on a periodic basis.

12. The computer-readable media of claim 7, wherein the aging the one or more counts occurs on an individual data entry basis.

13. The computer-readable media of claim 7, wherein the selecting the one or more aging factors is based on a piecewise continuous aging function.

14. A statistical classification system comprising:

a statistical model comprising a series of counts, with each count counting a number of occurrences of a feature value per class amongst a set of data entries;

a statistical classifier for generating the statistical model and using the statistical model to classify new data entries;

and a count modification mechanism, external to the statistical classifier, for modifying the series of counts.

15. The statistical classification system of claim 14, wherein the count modification mechanism comprises an interface for receiving modification data.

16. The statistical classification system of claim 14, wherein the count modification mechanism comprises a boosting mechanism for selecting one or more boost values for one or more counts of the series of counts.

17. The statistical classification system of claim 14, wherein the count modification mechanism comprises an aging mechanism for selecting one or more aging factors for one or more counts of the series of counts.

18. The statistical classification system of claim 17, wherein the series of counts comprises an aged series of counts and a non-aged series of counts.

19. The statistical classification system of claim 17, wherein the count modification mechanism further comprises a boosting mechanism for selecting one or more boost values for the one or more counts of the series of counts.

20. The statistical classification system of claim 17, wherein the aging mechanism selects the one or more aging factors by performing steps comprising:

selecting an initial set of one or more aging factors;

aging the one or more counts with the initial set of one or more aging factors to form an initial aged set of counts;

determining an initial predictive accuracy of the statistical classifier using the initial aged set of counts;

selecting a subsequent set of one or more aging factors;

aging the one or more counts with the subsequent set of one or more aging factors to form an subsequent aged set of counts;

determining a subsequent predictive accuracy of the statistical classifier using the subsequent aged set of counts;

comparing the initial predictive accuracy to the subsequent predictive accuracy; and

selecting the one or more aging factors based on a highest relative predictive accuracy of the statistical classifier using the aged one or more counts.