Computer-implementable Internet prediction method

• US 6,631,360 B1
• Filed: 11/06/2000
• Issued: 10/07/2003
• Est. Priority Date: 11/06/2000
• Status: Expired due to Term

• Current Assignee: BJ Hunter Information, Inc., Fujitsu Limited
• Original Assignee: Sightward, Inc. (Omaha Holdco Incorporated)
• Inventors: Cook, Rodney C.
• Primary Examiner(s): Davis, George B.
• Application Number:
  US09/707,547
• Time in Patent Office: 1,065 Days
• Field of Search: 706/20, 706/21, 700/83
• US Class Current: 706/20
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling

Abstract

A computer-implementable method of selecting which engine of a plurality of inference engines to use to predict the categories into which individuals fall, such as buyer/non-buyer, and produce forecast reports based on the predictions is disclosed. Training (known) sample data that categorizes individuals based on the individual'"'"'s profile is sequentially applied to multiple inference engines to determine which engine is best based on a desired objective. Then, a classifier associated with the selected engine is used to analyze unknown sample data, create category predictions and produce forecast reports based on the predictions.

Claims

1. A method of selecting which inference engine of a plurality of inference engines to use to create an estimated density function and a decision array for use in predicting the categories into which individuals fall and produce forecast reports based on the predictions, said method comprising:

a) a training process comprising

1) sequentially applying training sample data that categorizes individuals based on the individuals'"'"' profile features to multiple inference engines to determine which inference engine is best based on a desired objective;

2) creating an estimated density function and a density array for the best inference engine; and

3) storing the estimated density function and decision array for the best inference engine; and

b) an unknown sample data analysis process comprising;

1) using the stored estimated density function and decision array for the best inference engine to analyze unknown sample data that includes the profile features of individuals and predict the categories into which individuals fall; and

2) creating forecast reports based on the predicted categories.

2. The method claimed in claim 1 wherein:

a) the individual profile features included in the training sample data are sorted into sets of individual profile features; and

b) sequentially applying training sample data that categorizes individuals based on the individuals'"'"' profile features to multiple inference engines to determine which inference engine is best based on a desired objective includes;

1) selecting a set of features for a selected inference engine;

2) applying training sample data based on the selected set of features to the selected inference engine;

3) creating an estimated density function for the selected inference engine; and

4) repeating steps

     1) through

     3) until all sets of features have been selected.

3. The method claimed in claim 2 wherein the individual profile features are sorted into sets using standard statistics after controlling for multicolinearities.

4. The method claimed in claim 1 wherein the training process also comprises:

a) identifying a data source for each category;

b) establishing a training sample set; and

c) creating and storing a training data structure based on the training sample set, said training data structure categorizing individuals based on the individuals'"'"' profile features.

5. The method claimed in claim 4 wherein, after the training data structure is created and stored, the training sample data is analyzed by sequentially applying the training data structure to multiple inference engines to determine which engine is best based on a desired objective.

6. The method claimed in claim 5 wherein:

a) the individual profile features included in the training data structure are sorted into sets of individual profile features; and

b) sequentially applying the training data structure to multiple inference engines to determine which inference engine is best based on a desired objective includes;

1) selecting a set of features for a selected inference engine;

2) applying a training data structure based on the selected set of features to the selected inference engine;

3) creating an estimated density function for the selected inference engine; and

4) repeating

     1) through

     3) until all sets of features have been selected.

7. The method claimed in claim 6 wherein the individual profile features are sorted into sets using standard statistics after controlling for multicolinearities.

8. The method claimed in claim 1, wherein after the training sample data is analyzed by sequentially applying a training data structure to multiple inference engines to determine which engine is best based on a desired objective.

9. The method claimed in claim 8 wherein the training sample data is analyzed using a leaving-n-out approach.

10. The method claimed in claim 9 wherein the leaving-n-out approach comprises:

a) selecting a category;

b) removing n individuals'"'"' data from the training data structure;

c) estimating a density function and calculating a density value for each category based on the training data structure with the n individuals'"'"' data removed;

d) reinserting the n individuals'"'"' data into the training data structure; and

e) repeating b) through d) until all individuals have been processed.

11. The method claimed in claim 10 wherein after a density function has been estimated and a density value calculated for each category for each individual with n individuals'"'"' data removed, repeating a) through f) of claim 10 for the next category.

12. The method claimed in claim 11 wherein the training data structure analysis continues until all categories have been processed.

13. The method claimed in claim 12 wherein the calculated density values are used to create a density value data structure.

14. The method claimed in claim 13 wherein after the density value data structure is created, for each category and each individual, a decision rule is applied to the density value data structure for the individual and the results are used to create the decision array.

15. The method claimed in claim 14 wherein after the decision array is created, the decision array is displayed so that a user can determine if the objective has been met.

16. The method claimed in claim 15 wherein, based on user input, adjusting the decision surface of the desired objective and, for each category and each individual, applying a decision rule to the density value data structure for the individual and using the results to create a new decision array.

17. The method claimed in claim 1 including, in response to user input, adjusting the decision surface of the desired objective.

18. A computer readable medium for carrying out the method of any one of claims 1-17.

20. The method claimed in claim 19 wherein:

a) prior to performing the training process a set of individual profile features is selected;

b) after performing the calibration process a test is made to determine if other sets of individual features remain to be selected;

c) if other sets of features remain to be selected, selecting another set of features and performing c) and d) of claim 18; and

d) repeating a) through c) of this claim until all sets of sets of features have been selected.

19. A method of selecting which inference engine of a plurality of inference engines to use to create an estimated density function and a decision array for predicting the categories into individual falls and creating forecast based on the predictions, said method comprising:

a) setting up training sample data that categorizes individuals based on the individuals'"'"' profile features;

b) selecting an inference engine;

c) performing a training process on the training sample data to estimate a density function and create a density value data structure;

d) performing a calibration process on the density value data structure to create a decision array;

e) determining if another inference engine remain to be selected;

f) if any more inference engines remain to be selected, selecting another inference engine and repeating c) and d);

g) repeating b) through f) until all inference engines have been selected; and

h) using the estimated density function and the decision array to analyze an unknown data sample to create category predictions and forecasts based on the predictions.

21. The method claimed in claim 19 wherein setting up training sample data comprises:

a) selecting a category;

b) identifying a data source for the selected category;

c) determining if any more categories exist;

d) if any more categories exist, selecting the next category and identifying a data source for that category;

e) repeating b) through d) until no more categories exist;

f) when no more categories exist, establishing a training sample set; and

g) creating and storing a training data structure based on the training sample set.

22. The method claimed in claim 19 wherein setting up the training sample data creates a training data structure.

23. The method claimed in claim 22 wherein the training process comprises:

a) selecting a category;

b) selecting n individuals;

c) updating the training data structure by removing the selected n individuals from the selected category;

d) estimating a density function and calculating a density value;

e) updating the training data structure by adding back the selected n individuals to the selected category;

f) determining if any more individuals exist;

g) if more individuals exist, selecting the next n individuals and repeating c) through f) until no more individuals exist;

h) when no more individuals exist, determining if any more categories exist; and

i) if more categories exist, selecting the next category and repeating b) through h) until no more categories exist.

24. The method claimed in claim 23 wherein estimating the density function and calculating the density value comprise:

a) selecting a category;

b) estimating a density function for the selected category;

c) storing the density function estimate;

d) calculating a density value;

e) creating and storing a density value data structure;

f) determining if any more categories exist; and

g) if more categories exist, selecting the next category and repeating b) through f) until no more categories exist.

25. The method claimed in claim 24, wherein the calibration process comprises:

a) selecting a category;

b) selecting an individual;

c) applying a decision rule to the density value data structured for the individual;

d) adding the results to a decision array;

e) determining if any more individuals exist;

f) if more individuals exist, selecting the next individual and repeating c) through e) until no more individuals exist;

g) when no more individuals exist, determining if any more categories exist;

h) if more categories exist, selecting the next category and repeating b) through f) until no more categories exist;

i) when no more categories exist displaying the decision array so that a user can determine if the objective has been met;

j) if the user decides that the objective has been met, determining if the current decision array is better than a stored decision array; and

k) replacing the stored decision array and the estimated density function if the current decision array is better than the stored decision array.

26. The method claimed in claim 19 including setting up an unknown data sample.

27. The method claimed in claim 26 wherein setting up an unknown data sample comprises:

identifying a data source;

establishing an unknown data sample; and

creating and storing an unknown sample data structure.

28. The method claimed in claim 19 wherein category predictions and forecast reports are created by:

a) selecting an individual;

b) predicting the category into which the individual falls;

c) storing the prediction;

d) determining if any more individuals exist;

e) if more individuals exist, selecting the next individual and repeating b) through d) until all individuals have been selected; and

f) when all individuals have been selected creating a forecast report based on the predictions.

29. The method claimed in claim 19 including, in response to user input adjusting the decision surface of the desired objective.

30. A computer readable medium for carrying out the method of any one of claims 19-28.