CONVERGENT CONSTRUCTION OF TRADITIONAL SCORECARDS

• US 20080103999A1
• Filed: 10/17/2007
• Published: 05/01/2008
• Est. Priority Date: 10/07/2002
• Status: Active Grant

• Current Assignee: Neural Technologies Limited
• Original Assignee: Neural Technologies Limited
• Inventors: Peacock, Gavin, Bolt, George
• Granted Patent: US 7,577,624 B2
• Time in Patent Office: Days
• US Class Current: 706/15
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06Q 99/00 Subject matter not provided for in other groups of this subclass

Abstract

A neural model for simulating a scorecard comprises a neural network for transforming one or more inputs into an output. Each input of the neural model has a squashing function applied thereto for simulating a bin of the simulated scorecard. The squashing function includes a control variable for controlling the steepness of the response to the squashing function'"'"'s input so that during training of the neural model the steepness can be controlled. The output of the neural model represents the score of the simulated scorecard. The neural network is trained to behave like a scorecard by providing plurality of example values to the inputs of the neural network. Each output score produced is compared to an expected score to produce an error value. Each error value is back-propagated to adjust the neural network transformation to reduce the error value. The steepness of each squashing function is controlled using the respective control variable to affect the response of each squashing function.

Claims

1. A neural model for simulating a scorecard, comprising:

a neural network configured to transform one or more inputs into an output, each input of the neural model having a squashing function applied thereto for simulating a bin of the simulated scorecard, wherein the squashing function includes a control variable for controlling the steepness of the response to the squashing function'"'"'s input so that during training of the neural model the steepness can be controlled, the output representing the score of the simulated scorecard.

2. A neural model according to claim 1, wherein each input to the neural network represents a field with each field having one or more bins associated therewith.

3. A neural model according to claim 2, wherein the bins associated with the same field have the same control variable for controlling the response of the respective squashing functions.

4. A neural model according to claim 3, wherein the control variable associated with each field is independent of the control variable associated with the other fields.

5. A neural model according to claim 2, wherein each bin associated with the same field has a different offset applied to the input of the associated squashing function to differentiate one bin from another so that the output is allocated to the appropriate bin for that field.

6. A neural model according to claim 1, wherein one of the input fields is numeric.

7. A neural model according to claim 1, wherein one of the input fields is categorical, the categorical input field is encoded into binary inputs.

8. A neural model according to claim 7, wherein the categorical input field is hard coded into binary inputs.

9. A neural model according to claim 7, wherein the categorical input field is soft coded into binary inputs and post processed to provide a cut off for bin membership.

10. A neural model according to claim 1, wherein the neural network is arranged so that the squashing function steepness is of a low value during initial training and adjusted to be of a high value as the neural model reaches a state where the neural model behaves as the simulated scorecard.

11. A neural model according to claim 1, wherein a neural network is a multi-layered perceptron.

12. A neural model according to claim 1, wherein the squashing function is a sigmoid function.

13. A neural model according to claim 1, wherein the squashing function uses the following formula:

y=1/(1+exp(−

Tx))

where y is the result of the squashing function,

x is an input to the neural network,

T is the steepness control variable.

14. A neural model according to claim 1, wherein the score is calculated using the following formula:

$y_{\mathrm{num}}=\underset{i<br><br>\text{\hspace{1em}<br><br>}}{\overset{\text{\hspace{1em}<br><br>}}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{\Delta <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>s_i/\left(1+\exp <br><br>\left(-T<br><br>\left(x-{\mathrm{\beta <br><br>}}_i\right)\right)\right),$

where Y_num is the score,

i is a count variable for the number of bins,

β

_i is a bias of the ith bin boundary,

Δ

s_i is an amount added to the score by moving from bin i−

1 to bin i.

15. A method of training a neural network to behave like a scorecard, the neural network having one or more inputs and configured to transform the inputs into one or more outputs, each input having a squashing function applied thereto, each squashing function having a control variable for controlling the steepness of the response to the input of the squashing function, the method comprising:

providing a plurality of example values to the inputs of the neural network, each example producing an output representing a score;

comparing each score to an expected score of each example to produce an error value;

back-propagating each error value to adjust the neural network transformation to reduce the error value as each example is applied to the neural model; and

controlling the steepness of each squashing function using the respective control variable to affect the response of each squashing function.

16. A method according to claim 15, wherein each control variable is adjusted so that the respective steepness starts off low and ends high through the course of providing each example, comparing the output score and the expected score and back-propagating the error information.

17. A method according to claim 15, wherein the control variables are adjusted such that the respective steepness increased relative to how close the model is to a final state.

18. A method according to claim 15, wherein the training ends when one of the steepnesses rises above a threshold.

19. A method according to claim 15, wherein the training ends when all of the steepnesses rise above a threshold.

20. A method according to claim 15, wherein each input represents a field and each squashing function applied to each input simulates a scorecard bin, wherein one or more bins are associated with each field, and wherein the maximum number of bins per field is defined when the neural network is initialized.

21. A method according to claim 20, wherein a bin boundary between bins is removed if the disruption caused by removing the bin boundary is below a bin removal threshold.

22. A method according to claim 21, wherein in the event that a bin boundary is removed the steepness control variable associated with that field is adjusted to reduce the steepness.

23. A simulated scorecard apparatus comprising.

a neural network processor arranged to receive one or more inputs, and process the inputs to produce an output representing a score;

wherein the processor is configured to operate as a neural model with a squashing function applied to each of the inputs for simulating a bin of a simulated scorecard, each squashing function including a control variable for controlling the steepness of the response to the squashing function'"'"'s input,

wherein the processor is configured to be trained to simulate the scorecard in a trained state, such that in the trained state each steepness is high relative to the steepness of the neural model in an untrained state.

24. An apparatus according to claim 23, wherein each input to the processor represents a field of the simulated scorecard.

25. An apparatus according to claim 24, wherein the processor is configured to trigger one of a plurality of bins associated with each field and depending on the bin triggered in each field allocate a score for each field.

26. An apparatus according to claim 25, wherein the processor is configured to sum the scores for each field to calculate the score output as the result of the simulated scorecard.

27. An apparatus according to claim 24, wherein the processor is configured to apply an offset to each squashing function of each bin associated with the same field to differentiate one bin from another.

28. A trained neural model for simulating a scorecards, comprising:

a neural network configured to transform one or more inputs into an output representing a score;

wherein each input of the neural model has a squashing function applied thereto for simulating a bin of the simulated scorecard, the squashing function including a control variable for controlling the steepness of the response to the squashing function'"'"'s input,

wherein the steepness is high relative to the steepness of the neural network when it was untrained.

29. A neural model according to claim 28, wherein each input to the neural network represents a field with each field having one or bins associated therewith.

30. A neural model according to claim 29, wherein each bin associated with the same field has a different offset applied to the input of the associated squashing function to differentiate one bin from another, whereby the output is allocated to the appropriate bin for that field.

31. A system for training a neural network to behave like a scorecard, the neural network having one or more inputs and configured to transform the inputs into one or more outputs, each input having a squashing function applied thereto, each squashing function having a control variable for controlling the steepness of the response to the input of the squashing function, the system comprising:

means for providing a plurality of example values to the inputs of the neural network, each example producing an output representing a score;

means for comparing each score to an expected score of each example to produce an error value;

means for back-propagating each error value to adjust the neural network transformation to reduce the error value as each example is applied to the neural model; and

means for controlling the steepness of each squashing function using the respective control variable to affect the response of each squashing function.