Convergent construction of traditional scorecards

• US 7,577,624 B2
• Filed: 10/17/2007
• Issued: 08/18/2009
• Est. Priority Date: 10/07/2002
• Status: Expired due to Term

• Current Assignee: Neural Technologies Limited
• Original Assignee: Neural Technologies Limited
• Inventors: Peacock, Gavin, Bolt, George
• Primary Examiner(s): Hirl; Joseph P
• Application Number:
  US11/874,140
• Publication Number:
  US 20080103999A1
• Time in Patent Office: 671 Days
• Field of Search: None
• US Class Current: 706/17
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06Q 99/00 Subject matter not provided for in other groups of this subclass

Abstract

A neural model for simulating a scorecard comprises a neural network for transforming one or more inputs into an output. Each input of the neural model has a squashing function applied thereto for simulating a bin of the simulated scorecard. The squashing function includes a control variable for controlling the steepness of the response to the squashing function'"'"'s input so that during training of the neural model the steepness can be controlled. The output of the neural model represents the score of the simulated scorecard. The neural network is trained to behave like a scorecard by providing plurality of example values to the inputs of the neural network. Each output score produced is compared to an expected score to produce an error value. Each error value is back-propagated to adjust the neural network transformation to reduce the error value. The steepness of each squashing function is controlled using the respective control variable to affect the response of each squashing function.

Claims

1. A trainable neural network which is able to undergo training so as to result in a trained neural network operable as a scorecard, comprising:

a transformation layer transforming one or more received inputs into an output score, wherein the transformation layer is trainable so that when trained the score is determined according to the one or more inputs according to the training;

wherein the transformation layer comprises a computing component for each input applying one or more squashing functions to the respective input, each squashing function simulating a scorecard bin and having a control variable for controlling the steepness of a response by the squashing function to the respective input;

wherein each computing component is configured so that during training the respective control variable is varied, and at the completion of the training the respective control variable is fixed.

2. A neural network according to claim 1, wherein each computing component is configured so that the variation of each control variable is such that the steepness of the respective squashing function is gradually increased, until a modification occurs to one or more bins associated with one of the inputs, whereupon for those squashing functions associated with the one of the inputs has the control variable varied so that the steepness of the respective squashing function is decreased.

3. A neural network according to claim 1, wherein those squashing functions associated with the same input have the same control variable for controlling the steepness of the response to the respective input.

4. A neural network according to claim 3, wherein the control variable of one input is independent of the control variable of the other inputs.

5. A neural network according to claim 1, wherein each computing component is applying a different offset to the input for each bin prior to the application of the associated squashing function so as to differentiate one bin from another.

6. A neural network according to claim 1, wherein one of the inputs is categorical, and the categorical input is encoded into binary inputs to the one or more squashing functions.

7. A neural network according to claim 6, wherein the categorical input is hard coded into binary inputs to the one or more squashing functions.

8. A neural network according to claim 6, wherein the categorical input is soft coded into binary inputs to the one or more squashing functions and post processed to provide a cut off for bin membership.

9. A neural network according to claim 1, wherein the transformation layer is arranged so that the control variable is of a low value at the start of training and is adjusted to be of a high value when training is finished.

10. A neural network according to claim 1, wherein the score is calculated using the following formula:

$y_{\mathrm{num}}=\underset{i<br><br>}{\overset{}{\sum <br><br>}}<br><br><br><br>\mathrm{\Delta <br><br>}<br><br><br><br>s_i/\left(1+\exp <br><br>\left(-T<br><br>\left(x-{\mathrm{\beta <br><br>}}_i\right)\right)\right),$

where

y_num is the score,

i is a count variable for the number of bins,

x is a an input to the neural network,

T is the steepness control variable,

β

_i is a bias of the ith bin boundary, and

Δ

s_i is an amount added to the score by moving from bin i-1 to bin i.

11. A method of configuring a neural network to behave like a scorecard, the neural network comprising a transformation layer transforming one or more inputs into an output representing a score, wherein each input has one or more squashing functions applied thereto, each squashing function simulating a scorecard bin and having a control variable for controlling the steepness of a response by the squashing function to the respective input, the method comprising:

training the neural network, wherein during training the steepness of each control variable is varied and upon completion of the training each control variable is fixed.

12. A method according to claim 11, wherein each control variable is adjusted so that the respective steepness starts off low and ends high through the course of training the neural network.

13. A method according to claim 11, wherein the control variables are adjusted such that the respective steepness increases relative to how close the network is to a final trained state.

14. A method according to claim 11, wherein the training ends when one of the control variables rises above a threshold.

15. A method according to claim 11, wherein the training ends when all of the control variables rise above a threshold.

16. A method according to claim 11, wherein the number of bins associated with each input is defined when the training begins and the number of bins associated with each input is varied during training.

17. A method according to claim 16, wherein the number of bins is reduced by combining two adjacent bins if the disruption caused by removing a bin boundary between said adjacent bins is below a bin removal threshold.

18. A method according to claim 17, wherein in the event that a bin boundary is removed the control variable associated with an input associated with the combined bins is adjusted to reduce the steepness.

19. A trained neural network operating as a scorecard, comprising:

one or more receivers receiving one or more inputs;

an output; and

a transformation layer configured to transform the one or more inputs into a score provided to the output according to training conducted on the transformation layer,

wherein the transformation layer comprises a computing component for each input, each computing component configured such that a number of squashing functions are applied to the respective input, each squashing function simulating a scorecard bin and having a control variable for controlling the steepness of a response by the squashing function to the respective input,

wherein the transformation layer has been trained so that each control variable is fixed to a value as a result of training the transformation layer, wherein during training each control variable was varied and at the completion of the training the control variable of each input was fixed.

20. A neural network according to claim 19, wherein the number of squashing functions applied to each input is independent of the number of squashing functions applied to the other inputs and is determined as a result of training the neural network.

21. A neural network according to claim 20, wherein the transformation layer is triggering one of the one or more bins associated with each input and depending on which bin is triggered a contribution is made to the score.

22. A neural network according to claim 21, wherein the transformation layer is summing the score contributions to calculate the score output as the result of the simulated scorecard.

23. A neural network according to claim 20, wherein each computing component is applying a different offset to the input for each bin prior to application of the associated squashing function so as to differentiate one bin from another.

24. A neural network according to claim 19, wherein each computing component is applying a different scaling factor to the result of the squashing function for each bin so as to produce a different value according to the bin in which the input is simulated to fall within.

25. A system for training a neural network to behave like a scorecard, wherein the neural network is transforming one or more inputs into an output representing a score determined according the inputs, wherein each input has one or more squashing functions applied thereto, each squashing function for simulating a scorecard bin and having a control variable for controlling the steepness of a response by the squashing function to the respective input, the system comprising:

a controller for varying the steepness of each control variable during training of the neural network and upon completion of the training causing each control variable to be fixed.