Systems and methods for calibrating user and consumer data

• US 8,543,523 B1
• Filed: 03/15/2013
• Issued: 09/24/2013
• Est. Priority Date: 06/01/2012
• Status: Active Grant

• Current Assignee: comScore, Inc.
• Original Assignee: comScore, Inc.
• Inventors: Vinson, Michael, Goerlich, Bruce, Palit, Charles, Engel, Bill
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): Tran, Mai T
• Application Number:
  US13/836,591
• Time in Patent Office: 193 Days
• Field of Search: 706/12, 707/688, 707/752
• US Class Current: 706/12
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 30/0202 Market predictions or forecasting for commercial activities
• CPC Class Codes: G06Q 30/0203 Market surveys; Market polls
• CPC Class Codes: G06Q 30/0207 Discounts or incentives, e.g. coupons or rebates

Abstract

A system and method that calibrates subject data for which a relationship to a target population is not known, so that the calibrated subject data can more accurately represent the target population. In many cases the calibration will involve the use of a differential weighting scheme applied to the data at the constituent level. The system and method allows the values of the observed variables in the subject data set to be weighted so that their incidence is equivalent to that of a reference population represented by a reference data set, even if the variables used in the reference data set to make estimates for the reference population were not collected or measured for the subject data set.

Claims

1. A method in a computing system for calibrating a subject data set based on information from a reference data set, each data set containing a plurality of participants and associated transactional data, the method comprising:

using a data partitioning scheme, partitioning the reference data set into a plurality of reference data partitions, each of the plurality of reference data partitions having an associated transactional characteristic and no two reference data partitions sharing a participant in common;

using the data partitioning scheme, partitioning the subject data set into a plurality of subject data partitions, wherein;

each of the plurality of subject data partitions has an associated transactional characteristic that is the same as the transactional characteristic associated with the corresponding reference data partition or has a high degree of correspondence with the transactional characteristic associated with the corresponding reference data partition; and

no two subject data partitions of the plurality of subject data partitions share a participant in common;

calculating weights associated with each of the plurality of subject data partitions to adjust for subject data partitions that are under- or over-represented, the weights calculated to adjust a distribution of the plurality of subject data partitions to be the same as a distribution of the plurality of reference data set partitions;

calculating a statistic for each of the plurality of subject data partitions; and

adjusting, by the computing system, the calculated statistics by applying the calculated weight for each subject data partition to the calculated statistic for each subject data partition, the applied weights producing calibrated estimates of the statistics for the plurality of subject data partitions.

2. The method of claim 1, wherein each weight is determined by:

determining a reference portion by dividing the number of participants in a reference data partition by the total number of participants in the reference data set;

determining a subject portion by dividing the number of participants in a subject data partition by the total number of participants in the subject data set; and

dividing the first subject portion by the first reference portion.

3. The method of claim 1, wherein the weight is expressed as a percentage or as an absolute number.

4. The method of claim 1, further comprising preconditioning the reference data set to reduce bias or improve accuracy prior to partitioning the reference data set into a plurality of reference data partitions.

5. The method of claim 4, wherein the preconditioning is based on population census data and the preconditioning involves adjusting the reference data set so that it more closely resembles the plurality of participants represented by the population census data.

6. The method of claim 1, further comprising preconditioning the subject data set to reduce bias or improve accuracy prior to partitioning the subject data set into a plurality of subject data partitions.

7. The method of claim 1, further comprising generating an estimate for a variable contained in the reference data set that is not contained in the subject dataset by:

identifying a rate of occurrence of the variable in each of the plurality of reference data partitions; and

applying the identified rate of occurrence for the variable in each of the plurality of subject data partitions.

8. The method of claim 7, wherein the value of the variable for each subject data partition is expressed as a single value, an interval value, or a range of values for each subject data partition, each of the range of values having an associated probability.

9. The method of claim 1, wherein the transactional data represents financial transactions.

10. The method of claim 1, wherein the financial transactions represent transactions selected from the group consisting of transactions made with a credit card, online transactions, cash register transactions, frequent flier transactions, and loyalty program transactions.

11. The method of claim 1, wherein the transactional data represents television viewing behavior or web browsing behavior.

12. The method of claim 1, wherein the statistic is a count of the number of transactions or a count of the number of entities performing transactions.

13. The method of claim 1, further comprising:

combining the weights associated with each of the plurality of subject data partitions to calculate a total weight; and

assessing a degree of closeness between the subject data set and the reference data set based on the total weight.

14. A non-transitory computer-readable medium encoded with instructions that, when executed by a processor, perform a method in a computing system for calibrating a subject data set based on information from a reference data set, each data set containing a plurality of participants and associated transactional data, the method comprising:

using a data partitioning scheme, partitioning the reference data set into a plurality of reference data partitions, each of the plurality of reference data partitions having an associated transactional characteristic and no two reference data partitions sharing a participant in common;

using the data partitioning scheme, partitioning the subject data set into a plurality of subject data partitions, wherein;

each of the plurality of subject data partitions has an associated transactional characteristic that is the same as the transactional characteristic associated with the corresponding reference data partition or has a high degree of correspondence with the transactional characteristic associated with the corresponding reference data partition; and

no two subject data partitions of the plurality of subject data partitions share a participant in common;

calculating weights associated with each of the plurality of subject data partitions to adjust for subject data partitions that are under- or over-represented, the weights calculated to adjust a distribution of the plurality of subject data partitions to be the same as a distribution of the plurality of reference data set partitions;

calculating a statistic for each of the plurality of subject data partitions; and

adjusting, by the computing system, the calculated statistics by applying the calculated weight for each subject data partition to the calculated statistic for each subject data partition, the applied weights producing calibrated estimates of the statistics for the plurality of subject data partitions.

15. The non-transitory computer-readable medium of claim 14, wherein each weight is determined by:

determining a reference portion by dividing the number of participants in a reference data partition by the total number of participants in the reference data set;

determining a subject portion by dividing the number of participants in a subject data partition by the total number of participants in the subject data set; and

dividing the first subject portion by the first reference portion.

16. The non-transitory computer-readable medium of claim 14, wherein the weight is expressed as a percentage or as an absolute number.

17. The non-transitory computer-readable medium of claim 14, further comprising instructions that cause the computing system to generate an estimate for a variable contained in the reference data set that is not contained in the subject dataset by:

identifying a rate of occurrence of the variable in each of the plurality of reference data partitions; and

applying the identified rate of occurrence for the variable in each of the plurality of subject data partitions.

18. The non-transitory computer-readable medium of claim 17, wherein the value of the variable for each subject data partition is expressed as a single value, an interval value, or a range of values for each subject data partition, each of the range of values having an associated probability.

19. The non-transitory computer-readable medium of claim 14, wherein the transactional data represents financial transactions.

20. The non-transitory computer-readable medium of claim 14, wherein the financial transactions represent transactions selected from the group consisting of transactions made with a credit card, online transactions, cash register transactions, frequent flier transactions, and loyalty program transactions.

21. The non-transitory computer-readable medium of claim 14, wherein the transactional data represents television viewing behavior or web browsing behavior.

22. A method in a computing system for calibrating a subject data set based on information from a reference data set, each data set containing a plurality of participants, the method comprising:

using a data partitioning scheme, partitioning the reference data set into a plurality of reference data partitions, each of the plurality of reference data partitions having one or more variables;

using the data partitioning scheme, partitioning the subject data set into a plurality of subject data partitions, wherein;

each of the plurality of subject data partitions has one or more variables that are the same as the one or more variables associated with the corresponding reference data partition or have a high degree of correspondence to the one or more variables associated with the corresponding reference data partition;

calculating weights associated with each of the plurality of subject data partitions to adjust for subject data partitions that are under- or over-represented with respect to the distribution of the reference data partitions, the weights calculated to adjust a distribution of the plurality of subject data partitions to be the same as a distribution of the plurality of reference data set partitions;

calculating a statistic for each of the plurality of subject data partitions; and

adjusting, by the computing system, the calculated statistics by applying the calculated weight for each subject data partition to the calculated statistic for each subject data partition, the applied weights producing calibrated estimates of the statistics for the plurality of subject data partitions.

23. The method of claim 22, wherein each weight is determined by:

determining a reference portion by dividing the number of participants in a reference data partition by the total number of participants in the reference data set;

determining a subject portion by dividing the number of participants in a subject data partition by the total number of participants in the subject data set; and

dividing the first subject portion by the first reference portion.

24. The method of claim 22, further comprising preconditioning the subject data set to reduce bias or improve accuracy prior to partitioning the subject data set into a plurality of subject data partitions.

25. The method of claim 22, further comprising generating an estimate for a variable contained in the reference data set that is not contained in the subject dataset by:

identifying a rate of occurrence of the variable in each of the plurality of reference data partitions; and

applying the identified rate of occurrence for the variable in each of the plurality of subject data partitions.

26. The method of claim 25, wherein the value of the variable for each subject data partition is expressed as a single value, an interval value, or a range of values for each subject data partition, each of the range of values having an associated probability.

27. The method of claim 22, wherein the subject data set and the reference data set represent transactional data or behavioral data.

28. The method of claim 22, wherein the subject data set and the reference data set are associated with transactional data, and wherein the transactional data represent transactions selected from the group consisting of transactions made with a credit card, online transactions, cash register transactions, frequent flier transactions, and loyalty program transactions.

29. The method of claim 22, wherein the statistic is a count of the number of transactions or a count of the number of entities performing transactions.

30. The method of claim 22, further comprising:

combining the weights associated with each of the plurality of subject data partitions to calculate a total weight; and

assessing a degree of closeness between the subject data set and the reference data set based on the total weight.