METHOD AND APPARATUS FOR FACILITATING ON-DEMAND BUILDING OF PREDICTIVE MODELS

• US 20170185904A1
• Filed: 12/22/2016
• Published: 06/29/2017
• Est. Priority Date: 12/29/2015
• Status: Abandoned Application

• Current Assignee: 24/7 Customer Incorporated
• Original Assignee: 24/7 Customer Incorporated
• Inventors: PADMANABHAN, Dhanesh, RAJAGOPALAN, Ravishankar, ABBENA, Aravind Kumar Yadav, DANISH, Mohd Suaib, VISWANATHAN, Aditi, RAYCHAUDHURI, Samik
• Application Number:
  US15/389,170
• Publication Number:
  US 20170185904A1
• Time in Patent Office: Days
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06Q 10/04 Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem" market predictions or forecasting for commercial activities G06Q30/0202

Abstract

A computer-implemented method and an apparatus facilitate on-demand building of predictive models. Data corresponding to a plurality of customers is retrieved from data sources and a training data sample and a testing data sample are generated from the retrieved data. Variables for developing the predictive model are identified from the retrieved data and are subjected to any of structured categorical variable binning, unstructured categorical variable binning, and numeric variable binning to generate transformed variables. The transformed variables and the training data sample are used to develop predictive models. The developed predictive models are tested using the testing data sample and scores are generated corresponding to the developed predictive models. A predictive model is selected from among the developed predictive models based on the scores. The selected predictive model is published on a prediction platform to facilitate prediction of outcomes related to customers of the enterprise.

Claims

1. A computer-implemented method, comprising:

receiving, by a processor, from a user;

at least one specification for developing at least one predictive model, and

an input identifying one or more data sources storing data related to a plurality of customers of an enterprise;

retrieving, by the processor, the data related to the plurality of customers from the one or more data sources;

generating, by the processor, a training data sample and a testing data sample from the retrieved data;

performing, by the processor, at least one of structured categorical variable binning, unstructured categorical variable binning, and numeric variable binning;

generating, by the processor, transformed variables from variables identified for developing the at least one predictive model;

developing, by the processor, one or more predictive models based, at least in part, on the transformed variables and the training data sample;

generating, by the processor, at least one score corresponding to each predictive model from among the one or more predictive models based, at least in part, on the testing data sample;

selecting, by the processor, a predictive model from among the one or more predictive models based on the at least one score associated with each predictive model;

publishing the predictive model, by the processor, on a prediction platform; and

applying, by the processor, the published predictive model to predict outcomes related to customers of the enterprise.

2. The method of claim 1, wherein a specification from among the at least one specification comprises a problem definition for developing the at least one predictive model, the problem definition comprising an objective of the user for developing the at least one predictive model.

3. The method of claim 2, wherein the objective comprises any of customer intent prediction, spam or fraud detection, customer credit risk modelling, and customer lead scoring.

4. The method of claim 1, wherein a specification from among the at least one specification comprises a discrete outcome to be predicted for each customer among one or more customers of the enterprise by the at least one predictive model.

5. The method of claim 1, wherein a specification from among the at least one specification comprises one or more customer segments to be considered for prediction by the at least one predictive model.

6. The method of claim 5, wherein the specification corresponding to the one or more customer segments to be considered further comprises one or more pre-conditions for at least one customer segment from among the one or more customer segments.

7. The method of claim 1, wherein a specification from among the at least one specification comprises a sampling ratio for sampling the retrieved data to generate the training data sample and the testing data sample.

8. The method of claim 1, further comprising:

performing the identification of variables for developing the at least one predictive model based on one of user inputs specifying custom variables and pre-defined variables in the retrieved data.

9. The method of claim 1, further comprising:

subjecting one or more variables corresponding to structured categorical variables from among the identified variables to at least one clustering algorithm to perform the structured categorical variable binning, the structured categorical variable binning reducing cardinality associated with the structured categorical variables.

10. The method of claim 1, further comprising:

converting unstructured data from among the retrieved data to structured data using natural language language algorithms to facilitate the unstructured categorical variable binning.

11. The method of claim 1, further comprising:

subjecting one or more variables corresponding to numeric variables from among the identified variables to at least one of a statistical algorithm and a clustering algorithm to perform the numeric variable binning, the numeric variable binning reducing cardinality associated with the numeric variables.

12. The method of claim 1, wherein developing the one or more predictive models comprises:

receiving user input related to any of;

variables to be selected from among the transformed variables;

at least one type of machine learning algorithm to be used for developing the one or more predictive models; and

metrics to be evaluated corresponding to the one or more predictive models; and

training, by the processor, the at least type one machine learning algorithm based on the transformed variables, the training data sample, and the user input, the training of the at least one machine learning algorithm causing development of the one or more predictive models.

13. The method of claim 1, further comprising:

monitoring, by the processor, a real-time usage behavior of the published predictive model subsequent to the publishing of the predictive model on the prediction platform.

14. The method of claim 13, wherein monitoring the real-time usage behaviour comprises generating scores related to at least one metric from among Popularity Stability Index (PSI) and Variable Stability Index (VSI).

15. The method of claim 14, further comprising:

triggering, by the processor, retraining of the published predictive model when a generated score corresponding to one of the PSI and the VSI is less than a predefined threshold value.

16. The method of claim 1, further comprising:

facilitating on-demand building of predictive models by providing an automated prediction workflow to the user to implement and execute the steps of retrieving, generating, performing, developing, generating, and publishing.

17. The method of claim 16, further comprising:

executing each step of the automated prediction workflow the processor using one or more Application Programming Interfaces (APIs).

18. An apparatus, comprising:

an input/output (I/O) module configured to receive from a user;

at least one specification for developing at least one predictive model, and

an input identifying one or more data sources storing data related to a plurality of customers of an enterprise;

a communication interface communicably coupled with the I/O module, the communication interface configured to retrieve the data related to the plurality of customers from the one or more data sources;

at least one processor; and

a memory having stored therein machine executable instructions, that when executed by the at least one processor, cause the apparatus to;

generate a training data sample and a testing data sample from the retrieved data;

perform any of structured categorical variable binning, unstructured categorical variable binning, and numeric variable binning;

generate transformed variables from variables identified for developing the at least one predictive model;

develop one or more predictive models based, at least in part, on the transformed variables and the training data sample;

generate at least one score corresponding to each predictive model from among the one or more predictive models based, at least in part, on the testing data sample;

select a predictive model from among the one or more predictive models based on the at least one score associated with each predictive model;

publish the predictive model on a prediction platform; and

apply the published predictive model to predict outcomes related to customers of the enterprise.

19. The apparatus of claim 18, wherein a specification from among the at least one specification comprises a problem definition for developing the at least one predictive model, the problem definition comprising an objective of the user for developing the at least one predictive model.

20. The apparatus of claim 18, wherein a specification from among the at least one specification comprises a discrete outcome to be predicted corresponding to each customer from among one or more customers of the enterprise by the at least one predictive model.

21. The apparatus of claim 18, wherein a specification from among the at least one specification comprises one or more customer segments to be considered for prediction by the at least one predictive model, the specification further comprising one or more pre-conditions for at least one customer segment from among the one or more customer segments.

22. The apparatus of claim 18, wherein a specification from among the at least one specification comprises a sampling ratio for sampling the retrieved data to generate the training data sample and the testing data sample.

23. The apparatus of claim 18, wherein for performing the structured categorical variable binning, the apparatus is further caused to:

subject one or more variables corresponding to structured categorical variables from among the identified variables to at least one clustering algorithm, the structured categorical variable binning reducing cardinality associated with the structured categorical variables.

24. The apparatus of claim 18, wherein for performing the unstructured categorical variable binning, the apparatus is further caused to:

use natural language language algorithms to convert unstructured data from among the retrieved data to structured data.

25. The apparatus of claim 18, wherein for performing the numeric variable binning, the apparatus is further caused to:

subject one or more variables corresponding to numeric variables from among the identified variables to at least one of a statistical algorithm and a clustering algorithm, the numeric variable binning reducing cardinality associated with the numeric variables.

26. The apparatus of claim 18, wherein the apparatus is further caused to:

monitor a real-time usage behavior of the published predictive model subsequent to the publishing of the predictive model on the prediction platform; and

generate scores related to at least one metric from among Popularity Stability Index (PSI) and Variable Stability Index (VSI).

27. The apparatus of claim 26, wherein the apparatus is further caused to:

trigger retraining of the published predictive model when a generated score comprising one of the PSI and the VSI is less than a predefined threshold value.

28. An apparatus, comprising:

an input/output (I/O) module configured to receive from a user;

at least one specification for developing at least one predictive model, and

an input identifying one or more data sources storing data related to a plurality of customers of an enterprise;

a communication interface communicably coupled with the I/O module, the communication interface configured to facilitate retrieval of the data related to the plurality of customers from the one or more data sources;

a data ingestion module configured to generate a training data sample and a testing data sample from the retrieved data;

a transformation module configured to perform at least one of structured categorical variable binning, unstructured categorical variable binning, and numeric variable binning to generate transformed variables from variables identified for developing the at least one predictive model;

a model building module configured to develop one or more predictive models based, at least in part, on the transformed variables and the training data sample;

a model validating module configured to generate at least one score corresponding to each predictive model from among the one or more predictive models based, at least in part, on the testing data sample; and

a model publishing module configured to publish a predictive model on a prediction platform to facilitate prediction of outcomes related to customers of the enterprise, the predictive model selected from among the one or more predictive models based on the at least one score associated with each predictive model.

29. The apparatus of claim 28, wherein a specification from among the at least one specification comprises a problem definition for developing the at least one predictive model, the problem definition comprising an objective of the user for developing the at least one predictive model.

30. The apparatus of claim 28, wherein a specification from among the at least one specification comprises a discrete outcome to be predicted corresponding to each customer from among one or more customers of the enterprise by the at least one predictive model.

31. The apparatus of claim 28, further comprising:

a model monitoring module configured to monitor a real-time usage behavior of the published predictive model subsequent to the publishing of the predictive model on the prediction platform; and

a model retraining module configured to retrain the predictive model based on observed deviations during monitoring the real-time usage behavior of the published at least one predictive model.

32. The apparatus of claim 31, wherein each module from among the data ingestion module, the transformation module, the model building module, the model validation module, the model publishing module, the model monitoring module, and the model retraining module comprises one or more Application Programming Interfaces (APIs) to facilitate execution of respective tasks.

33. The apparatus of claim 32, the one or more APIs associated with the each module further comprising:

an automated prediction workflow provided to the user to facilitate on-demand building of predictive models.