Predictive Analytical Modeling Accuracy Assessment
First Claim
1. A computer-implemented system comprising:
- one or more computers; and
one or more data storage devices coupled to the one or more computers, storing;
a repository of training functions,a predictive model repository of trained predictive models, including a plurality of updateable trained predictive models, and wherein each trained predictive model is associated with an accuracy score that represents an estimation of the accuracy of the respective trained predictive model, andinstructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising;
receiving over a network a series of training data sets for predictive modeling from a client computing system, wherein training data included in the training data sets includes training samples that each comprise output data that corresponds to input data and wherein the training data included in the training data sets is different from initial training data that was used with a plurality of training functions obtained from the repository to train the trained predictive models stored in the predictive model repository;
upon receiving a first training data set included in the series of training data sets and for each trained predictive model in the predictive model repository, using the input data included in the first training data set to generate predictive output data and comparing the predictive output data to the output data included in the first training data set, and based on the comparison and previous comparisons that were determined from the initial training data and from previously received training data sets, determining an updated accuracy score for association with each trained predictive model in the repository;
for each updateable trained predictive model in the predictive model repository, using the first training data set, a first training function obtained from the repository of training functions that was used to generate the updateable trained predictive model and using the updateable trained predictive model, to generate a retrained predictive model and replacing the updateable trained predictive model in the predictive model repository with the retrained predictive model;
selecting a first trained predictive model from among the plurality of trained predictive models and retrained predictive models included in the predictive model repository based on the determined updated accuracy scores; and
providing access to the first trained predictive model over the network.
2 Assignments
0 Petitions
Accused Products
Abstract
A system includes a computer(s) coupled to a data storage device(s) that stores a training function repository and a predictive model repository that includes includes updateable trained predictive models each associated with an accuracy score. A series of training data sets are received, being training samples each having output data that corresponds to input data. The training data is different from initial training data that was used with training functions from the repository to train the predictive models initially. Upon receiving a first training data set included in the series and for each predictive model in the repository, the input data in the first training set is used to generate predictive output data that is compared to the output data. Based on the comparison and previous comparisons determined from the initial training data and from previously received training data sets, an updated accuracy score for each predictive model is determined.
-
Citations
18 Claims
-
1. A computer-implemented system comprising:
-
one or more computers; and one or more data storage devices coupled to the one or more computers, storing; a repository of training functions, a predictive model repository of trained predictive models, including a plurality of updateable trained predictive models, and wherein each trained predictive model is associated with an accuracy score that represents an estimation of the accuracy of the respective trained predictive model, and instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising; receiving over a network a series of training data sets for predictive modeling from a client computing system, wherein training data included in the training data sets includes training samples that each comprise output data that corresponds to input data and wherein the training data included in the training data sets is different from initial training data that was used with a plurality of training functions obtained from the repository to train the trained predictive models stored in the predictive model repository; upon receiving a first training data set included in the series of training data sets and for each trained predictive model in the predictive model repository, using the input data included in the first training data set to generate predictive output data and comparing the predictive output data to the output data included in the first training data set, and based on the comparison and previous comparisons that were determined from the initial training data and from previously received training data sets, determining an updated accuracy score for association with each trained predictive model in the repository; for each updateable trained predictive model in the predictive model repository, using the first training data set, a first training function obtained from the repository of training functions that was used to generate the updateable trained predictive model and using the updateable trained predictive model, to generate a retrained predictive model and replacing the updateable trained predictive model in the predictive model repository with the retrained predictive model; selecting a first trained predictive model from among the plurality of trained predictive models and retrained predictive models included in the predictive model repository based on the determined updated accuracy scores; and providing access to the first trained predictive model over the network. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. A computer-implemented method comprising:
-
receiving over a network a series of training data sets for predictive modeling from a client computing system, wherein training data included in the training data sets includes training samples that each comprise output data that corresponds to input data and wherein the training data included in the training data sets is different from initial training data that was used with a plurality of training functions obtained from a repository of training functions to train a plurality of trained predictive models stored in a predictive model repository wherein each trained predictive model is associated with an accuracy that indicates an accuracy of the trained predictive model in generating predictive outputs; upon receiving a first training data set included in the series of training data sets and for each trained predictive model in the predictive model repository, using the input data included in the first training data set to generate predictive output data and comparing the predictive output data to the output data included in the first training data set, and based on the comparison and previous comparisons that were determined from the initial training data and from previously received training data sets, determining an updated accuracy score for association with each trained predictive model in the repository; for each updateable trained predictive model in the predictive model repository, using the first training data set, a first training function obtained from the repository of training functions that was used to generate the updateable trained predictive model and using the updateable trained predictive model, to generate a retrained predictive model and replacing the updateable trained predictive model in the predictive model repository with the retrained predictive model; selecting a first trained predictive model from among the plurality of trained predictive models and retrained predictive models included in the predictive model repository based on the determined updated accuracy scores; and providing access to the first trained predictive model over the network. - View Dependent Claims (8, 9, 10, 11, 12)
-
-
13. A computer-readable storage device encoded with a computer program product, the computer program product comprising instructions that when executed on one or more computers cause the one or more computers to perform operations comprising:
-
receiving over a network a series of training data sets for predictive modeling from a client computing system, wherein training data included in the training data sets includes training samples that each comprise output data that corresponds to input data and wherein the training data included in the training data sets is different from initial training data that was used with a plurality of training functions obtained from a repository of training functions to train a plurality of trained predictive models stored in a predictive model repository wherein each trained predictive model is associated with an accuracy that indicates an accuracy of the trained predictive model in generating predictive outputs; upon receiving a first training data set included in the series of training data sets and for each trained predictive model in the predictive model repository, using the input data included in the first training data set to generate predictive output data and comparing the predictive output data to the output data included in the first training data set, and based on the comparison and previous comparisons that were determined from the initial training data and from previously received training data sets, determining an updated accuracy score for association with each trained predictive model in the repository; for each updateable trained predictive model in the predictive model repository, using the first training data set, a first training function obtained from the repository of training functions that was used to generate the updateable trained predictive model and using the updateable trained predictive model, to generate a retrained predictive model and replacing the updateable trained predictive model in the predictive model repository with the retrained predictive model; selecting a first trained predictive model from among the plurality of trained predictive models and retrained predictive models included in the predictive model repository based on the determined updated accuracy scores; and providing access to the first trained predictive model over the network. - View Dependent Claims (14, 15, 16, 17, 18)
-
Specification