PREDICTIVE MODELING OPTIMIZATION

• US 20170262769A1
• Filed: 09/09/2016
• Published: 09/14/2017
• Est. Priority Date: 03/14/2016
• Status: Active Grant

• Current Assignee: Business Objects Incorporated (SAP SE)
• Original Assignee: Business Objects Incorporated (SAP SE)
• Inventors: McShane, Alan, Huu, Jacques Doan, Abdelrahman, Ahmed, Carme, Antoine, Lamy, Bertrand, Maali, Fadi, Ouologuem, Laya, Caires, Milena, Dulian, Nicolas, Marcade, Erik
• Granted Patent: US 10,789,547 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06F 9/54 Interprogram communication
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition

Abstract

Techniques are described for identifying an input training dataset stored within an underlying data platform; and transmitting instructions to the data platform, the instructions being executable by the data platform to train a predictive model based on the input training dataset by delegating one or more data processing operations to a plurality of nodes across the data platform.

Claims

1. A computer-implemented method performed by at least one processor, the method comprising:

identifying, by the at least one processor, an input training dataset stored within a distributed processing platform that includes a plurality of sub-systems;

transmitting, by the at least one processor, instructions from a client application to the distributed processing platform to request that at least one of the plurality of sub-systems be executed to perform at least one data processing operation to determine a predictive model based on the input training dataset; and

providing, by the at least one processor, the predictive model to determine one or more results each associated with a probability of occurrence of a value in a dataset.

2. The computer-implemented method of claim 1, wherein the instructions are sent from the client application to the distributed processing platform through a unified client that includes a plurality of sub-clients each configured to interface with a respective sub-system of the distributed processing platform.

3. The computer-implemented method of claim 1, further comprising:

executing, by the at least one processor, at least one local data processing operation on the client application to determine the predictive model;

wherein the at least one local data processing operation accepts input including a set of results from the at least one data processing operation performed on the distributed processing platform.

4. The computer-implemented method of claim 1, wherein the distributed processing platform is a Hadoop platform.

5. The computer-implemented method of claim 1, wherein the at least one data processing operation including a data processing operation that is performed by a Spark sub-system of the plurality of sub-systems.

6. The computer-implemented method of claim 1, wherein the method is independent of data transfer of the input training dataset from the distributed processing platform.

7. The computer-implemented method of claim 1, wherein the at least one data processing operation includes calculating one or more statistics associated with the input training dataset to reduce a number of variables for generating the predictive model.

8. The computer-implemented method of claim 7, wherein the at least one data processing operation further includes re-calculating the one or more statistics based on the one or more results.

9. The computer-implemented method of claim 1, wherein the at least one data processing operation includes encoding data of the input training dataset including converting alphanumeric data to numeric data.

10. The computer-implemented method of claim 1, wherein the at least one data processing operation includes performing a co-variance matrix calculation and a matrix inversion calculation with respect to the input training dataset.

11. The computer-implemented method of claim 1, wherein the at least one data processing operation includes slicing the input training dataset to determine one or more slices, and scoring the predictive model with respect to the one or more slices.

12. The computer-implemented method of claim 1, wherein the at least one data processing operation includes iteratively evaluating a performance of the predictive model based on a structural risk minimization.

13. A system comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor, the memory storing instructions which, when executed by the at least one processor, cause the at least one processor to perform operations comprising;

identifying an input training dataset stored within a distributed processing platform that includes a plurality of sub-systems;

transmitting instructions from a client application to the distributed processing platform to request that at least one of the plurality of sub-systems be executed to perform at least one data processing operation to determine a predictive model based on the input training dataset; and

providing the predictive model to determine one or more results each associated with a probability of occurrence of a value in a dataset.

14. The system of claim 13, wherein the instructions are sent from the client application to the distributed processing platform through a unified client that includes a plurality of sub-clients each configured to interface with a respective sub-system of the distributed processing platform.

15. The system of claim 13, the operations further comprising:

executing at least one local data processing operation on the client application to determine the predictive model;

wherein the at least one local data processing operation accepts input including a set of results from the at least one data processing operation performed on the distributed processing platform.

16. The system of claim 13, wherein the distributed processing platform is a Hadoop platform.

17. One or more non-transitory computer-readable storage media storing instructions which, when executed by at least one processor, cause the at least one processor to perform operations comprising:

identifying an input training dataset stored within a distributed processing platform that includes a plurality of sub-systems;

transmitting instructions from a client application to the distributed processing platform to request that at least one of the plurality of sub-systems be executed to perform at least one data processing operation to determine a predictive model based on the input training dataset; and

providing the predictive model to determine one or more results each associated with a probability of occurrence of a value in a dataset.

18. The one or more non-transitory computer-readable storage media of claim 17, wherein the at least one data processing operation includes calculating one or more statistics associated with the input training dataset to reduce a number of variables for generating the predictive model.

19. The one or more non-transitory computer-readable storage media of claim 18, wherein the at least one data processing operation further includes re-calculating the one or more statistics based on the one or more results.

20. The one or more non-transitory computer-readable storage media of claim 17, wherein the at least one data processing operation includes performing a co-variance matrix calculation and a matrix inversion calculation with respect to the input training dataset.