Methods and systems for analyzing weirdness of variables

• US 8,832,120 B2
• Filed: 01/30/2012
• Issued: 09/09/2014
• Est. Priority Date: 01/30/2012
• Status: Active Grant

• Current Assignee: Mastercard International Incorporated (MasterCard Incorporated)
• Original Assignee: Mastercard International Incorporated (MasterCard Incorporated)
• Inventors: Jones, William C.
• Primary Examiner(s): Casanova, Jorge A
• Application Number:
  US13/361,530
• Publication Number:
  US 20130198206A1
• Time in Patent Office: 953 Days
• Field of Search: 705/1.1, 705/7.11, 705/7.36, 705/7.37, 705/14.17, 705/14.46, 705/35, 705/348, 705/7.28, 705/38, 707/749
• US Class Current: 707/749
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06Q 20/4016 involving fraud or risk level assessment in transaction processing

Abstract

A computer-based method of determining a weirdness score for variables within a data set is provided. The method includes receiving a selection of a first variable, wherein the first variable is defined by a measure, a time period, and a plurality of entities, calculating a plurality of parameters for the first variable, wherein each of the plurality of parameters is calculated based at least in part on a deviation of a measured value from a predicted value, calculating a rank for each of the plurality of parameters for the first variable, wherein the rank of each parameter is calculated relative to corresponding parameters calculated for all other variables in the data set having the same measure and time period as the first variable, and calculating a weirdness score for the first variable based at least in part on the calculated rank of each of the plurality of parameters.

Claims

1. A computer-based method of determining a weirdness score for variables within a data set, said method performed using a computer coupled to a database, said method comprising:

receiving a selection of a first variable included within the data set, wherein the first variable is defined by a measure, a time period, and a plurality of entities;

calculating a plurality of parameters for the first variable, wherein each of the plurality of parameters is calculated based at least in part on a deviation of a measured value of the first variable from a predicted value of the first variable;

calculating a rank for each of the plurality of parameters for the first variable by;

identifying all other variables in the data set that have the same measure as the first variable and the same time period as the first variable;

identifying, for each of the other variables, a plurality of parameters that correspond to the plurality of parameters for the first variable; and

for each parameter of the plurality of parameters for the first variable, ranking the parameter for the first variable relative to the corresponding parameters for the other variables; and

calculating a weirdness score for the first variable, wherein the weirdness score is indicative of how unexpected the measured value of the first variable is, and wherein the weirdness score depends on the calculated rank of each of the plurality of parameters for the first variable.

2. A computer-based method according to claim 1, wherein calculating a plurality of parameters comprises calculating a first parameter, a second parameter, and a third parameter for the first variable.

3. A computer-based method according to claim 2, wherein calculating a first parameter comprises calculating the first parameter using

FP=|MV−

PV|,

wherein FP is the first parameter, MV is the measured value of the first variable, and PV is the predicted value of the first variable.

4. A computer-based method according to claim 2, wherein calculating a second parameter comprises calculating the second parameter using

SP=(% var−

Avg % var)/σ

,

wherein SP is the second parameter, % var is a percentage variance of the first variable, Avg % var is an average percentage variance for previously recorded values of the first variable, and σ

is a standard deviation of the average percentage variance.

5. A computer-based method according to claim 2, wherein calculating a third parameter comprises calculating the third parameter using

TP=(% var−

Avg % var_peer)/σ

_peer

wherein TP is the third parameter, % var is a percentage variance of the first variable, Avg % var_peer is an average percentage variance for previously recorded values of a peer group of variables, and σ

_peer is a standard deviation of the average percentage variance for the peer group, and wherein the peer group of variables includes variables in the data set having the same measure and time period as the first variable, and at least one entity in common with the first variable.

6. A computer-based method according to claim 1, wherein calculating a weirdness score comprises calculating a weighted sum of the ranks of each of the plurality of parameters for the first variable.

7. A computer-based method according to claim 1, wherein receiving a selection of a first variable included within the data set comprises receiving a selection of the first variable included within a data set containing financial transaction data.

8. A computer-based method according to claim 1, wherein receiving a selection of a first variable included within the data set comprises receiving a selection of a plurality of variables included within the data set.

9. A computer-based method according to claim 8, further comprising:

sorting the plurality of variables based on their respective weirdness scores; and

displaying at least a portion of the sorted variables.

10. A computer system for determining a weirdness score for variables within a data set, said computer system comprising:

a memory device for storing data; and

a computing device comprising a processor, said computing device coupled to said memory device, said computing device configured to;

receive a selection of a first variable included within the data set, wherein the first variable is defined by a measure, a time period, and a plurality of entities;

calculate a plurality of parameters for the first variable, wherein each of the plurality of parameters is calculated based at least in part on a deviation of a measured value of the first variable from a predicted value of the first variable;

calculate a rank for each of the plurality of parameters for the first variable by;

identifying all other variables in the data set that have the same measure as the first variable and the same time period as the first variable;

identifying, for each of the other variables, a plurality of parameters that correspond to the plurality of parameters for the first variable; and

for each parameter of the plurality of parameters for the first variable, ranking the parameter for the first variable relative to the corresponding parameters for the other variables; and

calculate a weirdness score for the first variable, wherein the weirdness score is indicative of how unexpected the measured value of the first variable is, and wherein the weirdness score depends on the calculated rank of each of the plurality of parameters for the first variable.

11. A computer system according to claim 10, wherein to calculate a plurality of parameters said computing device is configured to calculate a first parameter, a second parameter, and a third parameter for the first variable.

12. A computer system according to claim 11, wherein to calculate a first parameter said computing device is configured to calculate the first parameter using

FP=|MV−

PV|,

wherein FP is the first parameter, MV is the measured value of the first variable, and PV is the predicted value of the first variable.

13. A computer system according to claim 11, wherein to calculate a second parameter said computing device is configured to calculate the second parameter using

SP=(% var−

Avg % var)/σ

,

wherein SP is the second parameter, % var is a percentage variance of the first variable, Avg % var is an average percentage variance for previously recorded values of the first variable, and σ

is a standard deviation of the average percentage variance.

14. A computer system according to claim 11, wherein to calculate a third parameter said computing device is configured to calculate the third parameter using

TP=(% var−

Avg % var_peer)/σ

_peer

wherein TP is the third parameter, % var is a percentage variance of the first variable, Avg % var_peer is an average percentage variance for previously recorded values of a peer group of variables, and σ

_peer is a standard deviation of the average percentage variance for the peer group, and wherein the peer group of variables includes variables in the data set having the same measure and time period as the first variable, and at least one entity in common with the first variable.

15. A computer system according to claim 10, wherein to calculate a weirdness score said computing device is configured to calculate a weighted sum of the ranks of each of the plurality of parameters for the first variable.

16. A computer system according to claim 10, wherein to receive a selection of a first variable included within the data set said computing device is configured to receive a selection of the first variable included within a data set containing financial transaction data.

17. A computer system according to claim 10, wherein to receive a selection of a first variable included within the data set said computing device is configured to receive a selection of a plurality of variables included within the data set.

18. A computer system according to claim 17, wherein said computing device is further configured to:

sort the plurality of variables based on their respective weirdness scores; and

display at least a portion of the sorted variables.

19. A computer program embodied on a non-transitory computer readable medium for determining a weirdness score for variables within a data set, said program comprises at least one code segment executable by a computer to instruct the computer to:

receive a selection of a first variable included within the data set, wherein the first variable is defined by a measure, a time period, and a plurality of entities;

calculate a plurality of parameters for the first variable, wherein each of the plurality of parameters is calculated based at least in part on a deviation of a measured value of the first variable from a predicted value of the first variable;

calculate a rank for each of the plurality of parameters for the first variable by;

identifying all other variables in the data set that have the same measure as the first variable and the same time period as the first variable;

identifying, for each of the other variables, a plurality of parameters that correspond to the plurality of parameters for the first variable; and

for each parameter of the plurality of parameters for the first variable, ranking the parameter for the first variable relative to the corresponding parameters for the other variables; and

calculate a weirdness score for the first variable, wherein the weirdness score is indicative of how unexpected the measured value of the first variable is, and wherein the weirdness score depends on the calculated rank of each of the plurality of parameters for the first variable.

20. A computer program according to claim 19, wherein to calculate a plurality of parameters said program comprises at least one code segment executable by the computer to instruct the computer to:

calculate a first parameter, a second parameter, and a third parameter for the first variable.

21. A computer program according to claim 20, wherein to calculate a first parameter said program comprises at least one code segment executable by the computer to instruct the computer to:

calculate the first parameter using

FP=|MV−

PV|,

wherein FP is the first parameter, MV is the measured value of the first variable, and PV is the predicted value of the first variable.

22. A computer program according to claim 20, wherein to calculate a second parameter said program comprises at least one code segment executable by the computer to instruct the computer to:

calculate the second parameter using

SP=(% var−

Avg % var)/σ

,

wherein SP is the second parameter, % var is a percentage variance of the first variable, Avg % var is an average percentage variance for previously recorded values of the first variable, and σ

is a standard deviation of the average percentage variance.

23. A computer program according to claim 20, wherein to calculate a third parameter said program comprises at least one code segment executable by the computer to instruct the computer to:

calculate the third parameter using

TP=(% var−

Avg % var_peer)/σ

_peer

wherein TP is the third parameter, % var is a percentage variance of the first variable, Avg % var_peer is an average percentage variance for previously recorded values of a peer group of variables, and σ

_peer is a standard deviation of the average percentage variance for the peer group, and wherein the peer group of variables includes variables in the data set having the same measure and time period as the first variable, and at least one entity in common with the first variable.

24. A computer program according to claim 19, wherein to calculate a weirdness score said program comprises at least one code segment executable by the computer to instruct the computer to:

calculate a weighted sum of the ranks of each of the plurality of parameters for the first variable.

25. A computer program according to claim 19, wherein to receive a selection of a first variable included within the data set said program comprises at least one code segment executable by the computer to instruct the computer to:

receive a selection of the first variable included within a data set containing financial transaction data.

26. A computer program according to claim 19, wherein to receive a selection of a first variable included within the data set said program comprises at least one code segment executable by the computer to instruct the computer to:

receive a selection of a plurality of variables included within the data set.

27. A computer program according to claim 26, wherein said program comprises at least one code segment executable by the computer to instruct the computer to:

sort the plurality of variables based on their respective weirdness scores; and

display at least a portion of the sorted variables.

28. A network-based system for determining a weirdness score for variables within a data set, said system comprising:

a client computer system;

a database; and

a server system coupled to said client computer system and said database, said server system configured to;

receive from said client computer system a selection of a first variable included within the data set, wherein the first variable is defined by a measure, a time period, and a plurality of entities;

calculate a plurality of parameters for the first variable, wherein each of the plurality of parameters is calculated based at least in part on a deviation of a measured value of the first variable from a predicted value of the first variable;

calculate a rank for each of the plurality of parameters for the first variable by;

identifying all other variables in the data set that have the same measure as the first variable and the same time period as the first variable;

identifying, for each of the other variables, a plurality of parameters that correspond to the plurality of parameters for the first variable; and

for each parameter of the plurality of parameters for the first variable, ranking the parameter for the first variable relative to the corresponding parameters for the other variables; and

calculate a weirdness score for the first variable, wherein the weirdness score is indicative of how unexpected the measured value of the first variable is, and wherein the weirdness score depends on the calculated rank of each of the plurality of parameters for the first variable.