Fast explanations of scored observations

• US 5,745,654 A
• Filed: 02/13/1996
• Issued: 04/28/1998
• Est. Priority Date: 02/13/1996
• Status: Expired due to Term

• Current Assignee: Fair Isaac Corporation
• Original Assignee: HNC Software Inc. (Fair Isaac Corporation)
• Inventors: Titan, Hari
• Primary Examiner(s): Hafiz, Tariq R.
• Assistant Examiner(s): Rhodes, Jason W.
• Application Number:
  US08/600,858
• Time in Patent Office: 805 Days
• Field of Search: 395/22, 395/26, 395/21, 395/23, 395/11, 395/76, 395/51
• US Class Current: 706/20
• CPC Class Codes: G06N 3/105 Shells for specifying net layout

Abstract

A system, method, and product provide rapid explanations for the scores determined by a neural network for new observations input into the neural network. The explanations are associated with a table of percentile bins for each of the input variables used to define the observation. The table contains for each input variable a number of percentile bins. Each percentile bin contains an expected score for values of the input variable containing with the percentile bin. The expected score in each percentile bin is determined from historical observation data. Preferably each percentile bin is associated with an explanation that describes the meaning of the value of the input variable falling within the percentile bin. During observation processing, a new observation is scored. The value of each input variable in the new observation is compared with the percentile bins for the input variable in the table. The expected score in the percentile bin that contains the value of the input variable is retrieved, and this is repeated for all input variables in the new observation. The explanation associated with the percentile bin that has an expected score closest to the actual score is retrieved and provided as the explanation of the most significant input variable accounting for score. Other explanations from the next closest expected scores may also be retrieved.

Claims

1. A computer system for providing explanations of scored values for observations, comprising:

a database storing a plurality of observations, each observation having a value for each of a plurality of same input variables, and a score value for an output variable produced by a neural network;

a neural network that receives observations and produces an output score for the observation based on the values of the input variables;

a computer readable memory including;

a table including for each of the input variables, a set of a variable number of percentile bins, each percentile bin of an input variable associated with an expected score value for the output variable, the expected score value determined by an average of score values of observations having values for the input variable within the percentile bin, at least one percentile bin of each input variable associated with an explanation of the percentile bin;

a computer program that receives a new observation and score value from the neural network and determines from the table the percentile bin with an expected score value that is closest to the score value of the new observation, and provides the explanation, if any associated, with the determined percentile bin to a user.

2. In a computer system including a database storing a plurality of observations, each observation having a value for selected ones of a plurality of input variables, and a score value for an output variable determined with respect to the values of the input variable, a computer implemented method of providing explanations for the score values for new observations, comprising:

storing a table including for each of the input variables, a variable number of percentile bins, each percentile bin of an input variable associated with an expected score value for the output variable, the expected score value determined as a function of score values of observations having values for the input variable within the percentile bin;

for each input variable, associating with each of selected ones of the percentile bins of the input variable, an explanation of the values for the input variable within the percentile bin;

receiving a new observation;

determining a score value for the new observation;

determining from the table the percentile bin with an expected score value that is closest to the score value of the new observation; and

providing to a user the explanation, if any, associated with the determined percentile bin.

3. The method of claim 2, wherein storing a table includes:

for each input variable;

ranking the score values of the observations according to the value of the input variable for the observation;

determining a number of percentile bins for the input variable, each percentile bin having a ranking; and

determining for each percentile bin an expected score value for the output variable for all observations having values of the input variable within the percentile bin.

4. The method of claim 3, further comprising:

determining whether the number of observations in a lowest ranked bin is less than a minimum number of observations; and

responsive to the number of observations being less than the minimum number of observations, merging the lowest ranked bin with a second lowest ranked bin, and redetermining the expected score value for the output variable for all observations having values of the input variable within the merged percentile bins.

5. The method of claim 2, further comprising:

receiving for each observation a value of target variable;

for each input variable;

determining for each percentile bin of the input variable an expected target value for the target variable for all observations having values of the input variable within the percentile bin;

determining whether the expected score value for the output variable substantially approximates the expected target value;

responsive to the expected score value not substantially approximating the expected target value, either;

increasing the number of observations and redetermining the expected scores for each percentile bin;

or,

decreasing the number of percentile bins and redetermining the expected scores for each remaining percentile bin.

6. The method of claim 2 wherein determining from the table the percentile bin with an expected score value that is closest to the score value of the new observation comprises:

receiving the observation;

for each input variable in the observation;

obtaining an expected score from the percentile bin of the input variable within the table that contains the value of the input variable; and

selecting from the obtained expected scores the expected score that is closest to the score value of the new observation.

7. The method of claim 6, further comprising:

selecting from the obtained expected scores, the expected scores that are within a threshold value of the actual score; and

providing for each of the selected scores, an explanation associated with the percentile bin from which the expected score was obtained to explain each of the selected scores.

8. In a computer system including a database storing a plurality of observations, each observation having a value for selected ones of a plurality of input variables, and a score value for an output variable determined with respect to the values of the input variable, a computer implemented method of providing explanations for the score values for new observations, comprising:

creating a table including for each of the input variables, a set of a variable number of percentile bins, each percentile bin of an input variable associated with an expected score value for the output variable, the expected score value determined by averaging score values of observations having values for the input variable within the percentile bin, wherein the table is created by;

for each input variable;

ranking the score values of the observations according to the value of the input variable for the observation;

determining a number of percentile bins for the input variable, each percentile bin having a ranking;

determining for each percentile bin an expected score value for the output variable for all observations having values of the input variable within the percentile bin; and

determining whether the percentile bin is to be associated with an explanation, and if so, associating the percentile bin with an explanation;

receiving a new observation;

determining a score value for the new observation;

determining from the table the percentile bin with an expected score value that is closest to the score value of the new observation;

providing to a user the explanation, if any, associated with the determined percentile bin;

receiving for each observation a value of a target variable; and

for each input variable;

determining for each percentile bin of the input variable an expected target value for the target variable for all observations having values of the input variable within the percentile bin;

determining whether the expected score value for the output variable substantially approximates the expected target value;

responsive to the expected score value not substantially approximating the expected target value, either;

increasing the number of observations and redetermining the expected scores for each percentile bin;

or,

decreasing the number of percentile bins and redetermining the expected scores for each remaining percentile bin.

9. In a computer system including a database storing a plurality of observations, each observation having a value for selected ones of a plurality of input variables, and a score value for an output variable determined with respect to the values of the input variable, a computer implemented method of providing explanations for the score values for new observations, comprising:

creating a table including for each of the input variables, a set of a variable number of percentile bins, each percentile bin of an input variable associated with an expected score value for the output variable, the expected score value determined by averaging score values of observations having values for the input variable within the percentile bin, wherein the table is created by;

for each input variable;

ranking the score values of the observations according to the value of the input variable for the observation;

determining a number of percentile bins for the input variable, each percentile bin having a ranking;

determining for each percentile bin an expected score value for the output variable for all observations having values of the input variable within the percentile bin; and

determining whether the percentile bin is to be associated with an explanation, and if so, associating the percentile bin with an explanation;

receiving a new observation;

determining a score value for the new observation;

determining from the table the percentile bin with an expected score value that is closest to the score value of the new observation by;

for each input variable in the new observation, obtaining an expected score from the percentile bin of the input variable within the table that contains the value of the input variable;

selecting from the obtained expected scores the expected score that is closest to the score value of the new observation;

selecting the percentile bin having the closest expected score; and

providing to a user the explanation, if any, associated with the selected percentile bin.

10. The method of claim 9, further comprising:

selecting from the obtained expected scores, the expected scores that are within a threshold value of the actual score; and

providing for each of the selected scores, an explanation associated with the percentile bin from which the expected score was obtained.

11. In a computer system including a database storing a plurality of observations, each observation having a value for selected ones of a plurality of input variables, and a score value for an output variable determined with respect to the values of the input variable, a computer readable memory storing a computer program therein for configuring a processor of the computer system to create a table of percentile bins associated with explanations for the input variables for providing explanations for the score values for new observations, the computer program configuring the processor to:

provide a score value for the output variable of a selected number of observations based on the values of the input variables of each observation; and

for each input variable;

rank the score values of the observations according to the value of the input variable for the observation;

determine a number of percentile bins for the input variable, each percentile bin having a ranking;

determine for each percentile bin an expected score value for the output variable for all observations having values of the input variable within the percentile bin; and

determine whether the percentile bin is to be associated with an explanation, and if so, associate the percentile bin with an explanation of the input variable for the percentile bin and the values of the input variable within the percentile bin.

12. The computer readable memory of claim 11, further configuring the processor to:

determine whether the number of observations in a lowest ranked bin is less than a minimum number of observations; and

responsive to the number of observations being less than the minimum number of observations, merge the lowest ranked bin with a second lowest ranked bin, and redetermine the expected score value for the output variable for all observations having values of the input variable within the merged percentile bins.

13. A computer readable memory, for use with a processor and a database storing a plurality of observations, each observation having a value for selected ones of a plurality of same input variables, and a score value for an output variable determined with respect to the values of the input variable;

and a table including for each of the input variables, a set of a variable number of percentile bins, each percentile bin of an input variable associated with an expected score value for the output variable, the expected score value determined as a function of score values of observations having values for the input variable within the percentile bin, at least one percentile bin of each input variable in the table associated with an explanation of the input variables and the values of the input variable within the percentile bin, the computer readable memory storing a computer program for configuring and controlling the processor to provide explanations for the score values for new observations by performing the steps of;

receiving a new observation;

determining a score value for the new observation;

determining from the table the percentile bin with an expected score value that is closest to the score value of the new observation; and

providing to a user the explanation, if any, associated with the determined percentile bin.

14. A computer system for identifying at least one input variable significantly contributing to a score value for an individual observation, the system comprising:

a database storing a plurality of training observations, each training observation having a value for each of a plurality of input variables, and a score value for an output variable;

a scoring module that receives observations, and produces a score value for the output variable for the observation based on values of the input variables for the observation; and

a computer readable memory including;

a table including for each of the input variables, a variable number of percentile bins, each percentile bin of an input variable associated with an expected score value for the output variable, the expected score value determined as a function of score values of training observations having values for the input variable within the percentile bin; and

a computer program that receives a new, individual observation and score value from the scoring module and selects from the table the percentile bin with an expected score value that is closest to the score value of the new, individual observation, and identifies the input variable associated with the selected percentile bin.

15. In a computer system including a database storing a plurality of observations, each observation having a value for selected ones of a plurality of input variables, and a score value for an output variable determined with respect to the values of the input variable, a computer implemented method of identifying at least one input variable that significantly contributes to the score value for a new observation, comprising:

storing a table including for each of the input variables, a variable number of percentile bins, each percentile bin of an input variable associated with an expected score value for the output variable, the expected score value determined as a function of score values of observations having values for the input variable within the percentile bin;

receiving a new observation;

determining a score value for the new observation;

selecting from the table the percentile bin with an expected score value that is closest to the score value of the new observation; and

identifying the input variable associated with the selected percentile bin.

16. A computer implemented method of identifying an input variable that significantly contributes to a score value of an observation, comprising:

receiving a plurality of training observations, each training observation having a value for each input variable, and a score value of an output variable determined as a function of the values of the input variables;

for each input variable;

sorting the training observations according to the value of the input variable;

segregating the sorted training observations into subsets;

determining for each subset, an expected score value determined as a function of the score values of the training observations in the subset;

receiving a new observation having values for the input variables;

scoring the new observation to produce a new score value; and

comparing the new score value with the expected score values of the subsets of training observations for the input variables, to identify the input variable having a subset of training observations with an expected score value closest to the new score value of the new observation.

17. The computer implemented method of claim 16, further comprising:

for each input variable, associating with each subset of training scores, an explanation of the input variable and the values of the input variable in the subset; and

responsive to identifying the input variable having a subset of training observations with an expected score value closest to the new score value of the new observation, providing to a user the explanation associated with the subset of training observations.

18. In a computer system including a database storing a plurality of observations, each observation having a value for selected ones of a plurality of input variables, and a neural network score value for an output variable determined with respect to the values of the input variable, a computer implemented method of providing explanations for the neural network score values for new observations, comprising:

storing a table including for each of the input variables, a variable number of percentile bins, each percentile bin of an input variable associated with an expected score value for the output variable, the expected score value determined as a function of neural network score values of observations having values for the input variable within the percentile bin;

for each input variable, associating with each of selected ones of the percentile bins of the input variable, an explanation of the values for the input variable within the percentile bin;

receiving a new observation;

determining a neural network score value for the new observation;

determining from the table the percentile bin with an expected score value that is closest to the neural network score value of the new observation; and

providing to a user the explanation, if any, associated with the determined percentile bin.

19. A computer readable memory, for use with a processor and a database storing a plurality of observations, each observation having a value for selected ones of a plurality of same input variables, and a neural network score value for an output variable determined with respect to the values of the input variable, the computer readable memory storing a computer program for configuring and controlling the processor to create a table of percentile bins associated with explanations for the input variables, for providing explanations for the neural network score values for new observations by performing the steps of:

providing a neural network score value for the output variable of a selected number of observations based on the values of the input variables of each observation;

for each input variable;

ranking the neural network score values of the observations according to the value of the input variable for the observation;

determining a number of percentile bins for the input variable, each percentile bin having a ranking;

determining for each percentile bin an expected neural network score value for the output variable for all observations having values of the input variable within the percentile bin; and

determining whether the percentile bin is to be associated with an explanation, and if so, associate the percentile bin with an explanation of the input variable for the percentile bin and the values of the input variable within the percentile bin.

20. A computer readable memory, for use with a processor and a database storing a plurality of observations, each observation having a value for selected ones of a plurality of same input variables, and a neural network score value for an output variable determined with respect to the values of the input variable;

and a table including for each of the input variables, a set of a variable number of percentile bins, each percentile bin of an input variable associated with an expected neural network score value for the output variable, the expected neural network score value determined as a function of score values of observations having values for the input variable within the percentile bin, at least one percentile bin of each input variable in the table associated with an explanation of the input variables and the values of the input variable within the percentile bin, the computer readable memory storing a computer program for configuring and controlling the processor to provide explanations for the neural network score values for new observations by performing the steps of;

receiving a new observation;

determining a neural network score value for the new observation;

determining from the table the percentile bin with an expected neural network score value that is closest to the neural network score value of the new observation; and

providing to a user the explanation, if any, associated with the determined percentile bin.

21. In a computer system including a database storing a plurality of observations, each observation having a value for selected ones of a plurality of input variables, and a neural network score value for an output variable determined with respect to the values of the input variable, a computer implemented method of identifying at least one input variable that significantly contributes to the neural network score value for a new observation, comprising:

storing a table including for each of the input variables, a variable number of percentile bins, each percentile bin of an input variable associated with an expected score value for the output variable, the expected score value determined as a function of neural network score values of observations having values for the input variable within the percentile bin;

receiving a new observation;

determining a neural network score value for the new observation;

selecting from the table the percentile bin with an expected score value that is closest to the neural network score value of the new observation; and

identifying the input variable associated with the selected percentile bin.