System and method for prediction using synthetic features and gradient boosted decision tree

• US 10,366,451 B2
• Filed: 01/27/2016
• Issued: 07/30/2019
• Est. Priority Date: 01/27/2016
• Status: Active Grant

• Current Assignee: Huawei Cloud Computing Technologies Company Limited (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Kaznady, Max S
• Primary Examiner(s): Rankins, William E
• Application Number:
  US15/007,593
• Publication Number:
  US 20170213280A1
• Time in Patent Office: 1,280 Days
• Field of Search: 705 38, 705 35, 705 11
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 40/03 Credit; Loans; Processing thereof

Abstract

A machine learning system and method are disclosed in which a plurality of synthetic features are created from input data, and a gradient boosted decision tree algorithm is then executed by the computer to process both the synthetic features and at least some of the input data to produce an output that is a probability.

Claims

1. A computer-implemented method comprising:

the computer obtaining a set of data relating to a loan application, wherein the data includes an amount of loan requested and a transaction history of a loan applicant;

the computer determining a plurality of synthetic features by at least;

executing a plurality of machine learning algorithms that have been trained, each of the machine learning algorithms, when executed, receiving as an input at least some of the data and producing as an output a respective synthetic feature representing an initial probability of whether a loan default will occur, wherein at least two of the machine learning algorithms are different from each other and accept different inputs;

wherein a first one of the machine learning algorithms is implemented using a neural network and accepts the transaction history of the loan applicant as its input and outputs a first synthetic feature representing a first initial probability of whether the loan default will occur, and wherein a second one of the machine learning algorithms accepts the amount of loan requested as its input and outputs a second synthetic feature representing a second initial probability of whether the loan default will occur;

the computer executing a gradient boosted decision tree (GBDT) algorithm, the GBDT algorithm processing both;

(i) the synthetic features including the first synthetic feature and the second synthetic feature, and (ii) at least some of the data, and producing an output representing a final probability of whether the loan default will occur;

the computer generating an indication of whether or not to approve the loan based on whether a particular value is above or below a stored threshold, wherein the particular value is the final probability or is a function of the final probability.

2. The computer-implemented method of claim 1, wherein the plurality of machine learning algorithms are first machine learning algorithms, and wherein the computer-implemented method further comprises:

the computer executing at least one other machine learning algorithm different from the first machine learning algorithms, the at least one other machine learning algorithm, when executed, receiving an input based on at least some of the data and producing a plurality of outputs;

wherein each one of the plurality of outputs is a synthetic feature representing a probability of an event occurring, the event being different from the loan default that is associated with the final probability.

3. The computer-implemented method of claim 2, wherein the event comprises late payment of the loan.

4. The computer-implemented method of claim 1, further comprising:

the computer performing binary encoding of at least some of the data to produce binary encoded data; and

the computer inputting the binary encoded data to at least one of the machine learning algorithms.

5. The computer-implemented method of claim 1, further comprising:

the computer augmenting the data with at least one additional feature; and

the computer inputting the at least one additional feature to at least one of the machine learning algorithms.

6. The computer-implemented method of claim 1, wherein the data further includes a loan duration.

7. The computer-implemented method of claim 1, further comprising:

the computer training the plurality of machine learning algorithms, the training including using training data and test data to determine what inputs are to be used for training each machine learning algorithm by;

the computer trying different possible inputs and selecting a set of one or more inputs that best satisfy a metric.

8. A system comprising:

a memory to store a set of data relating to a loan application, wherein the data includes an amount of loan requested and a transaction history of a loan applicant;

a predictor to receive the data and to produce an output representing a final probability of whether a loan default will occur;

the predictor including a plurality of learners that have been trained, each learner implementing a respective machine learning algorithm, and wherein at least two of the learners implement a machine learning algorithm different from one another that accepts inputs different from one another;

the predictor configured to;

determine a plurality of synthetic features by sending to each of the learners at least some of the data, and each of the learners outputting a respective synthetic feature representing an initial probability of whether the loan default will occur, wherein a first learner of the learners implements a neural network and accepts the transaction history of the loan applicant and outputs a first synthetic feature representing a first initial probability of whether the loan default will occur, and wherein a second learner of the learners accepts the amount of loan requested as its input and outputs a second synthetic feature representing a second initial probability of whether the loan default will occur; and

execute a gradient boosted decision tree (GBDT) algorithm, the GBDT algorithm processing both;

(i) the synthetic features including the first synthetic feature and the second synthetic feature, and (ii) at least some of the data, and producing the output representing the final probability of whether the loan default will occur;

wherein the system is configured to generate an indication of whether or not to approve the loan based on whether a particular value is above or below a stored threshold, wherein the particular value is the final probability or is a function of the final probability.

9. The system of claim 8, wherein the plurality of learners are first learners, and wherein the predictor is configured to perform:

sending an input to at least one other learner different from the first learners, the input based on at least some of the data, and the at least one other learner implementing a machine learning algorithm that, when executed, receives the input and produces a plurality of outputs;

wherein each one of the plurality of outputs is a synthetic feature representing a probability of an event occurring, the event being different from the loan default that is associated with the final probability.

10. The system of claim 9, wherein the event comprises late payment of the loan.

11. The system of claim 8, wherein the predictor is further configured to:

perform binary encoding of at least some of the data to produce binary encoded data; and

send the binary encoded data to at least one of the learners.

12. The system of claim 8, wherein the predictor is further configured to:

augment the data with at least one additional feature; and

send the at least one additional feature to at least one of the learners.

13. The system of claim 8, wherein the data further includes a loan duration.

14. The system of claim 8, wherein the system is configured to train the plurality of learners, the training comprising using training data and test data to determine what inputs are to be sent to each learner by:

the system trying different possible inputs and selecting a set of one or more inputs that best satisfy a metric.

15. A system comprising:

at least one processor; and

memory having stored thereon processor-executable instructions that, when executed, cause the at least one processor to;

obtain a set of data relating to a loan application, wherein the data includes an amount of loan requested and a transaction history of a loan applicant;

determine a plurality of synthetic features by at least;

executing a plurality of machine learning algorithms that have been trained, each of the machine learning algorithms, when executed, receiving as an input at least some of the data and producing as an output a respective synthetic feature representing an initial probability of whether a loan default will occur, wherein at least two of the machine learning algorithms are different from each other and accept different inputs;

wherein a first one of the machine learning algorithms is implemented using a neural network and accepts the transaction history of the loan applicant and outputs a first synthetic feature representing a first initial probability of whether the loan default will occur, and wherein a second one of the machine learning algorithms accepts the amount of loan requested as its input and outputs a second synthetic feature representing a second initial probability of whether the loan default will occur; and

execute a gradient boosted decision tree (GBDT) algorithm, the GBDT algorithm processing both;

(i) the synthetic features including the first synthetic feature and the second synthetic feature, and (ii) at least some of the data, and producing an output representing a final probability of whether the loan default will occur;

generate an indication of whether or not to approve the loan based on whether a particular value is above or below a stored threshold, wherein the particular value is the final probability or is a function of the final probability.