System and methods for calibrating user and consumer data

• US 9,519,910 B2
• Filed: 09/19/2013
• Issued: 12/13/2016
• Est. Priority Date: 06/01/2012
• Status: Active Grant

• Current Assignee: comScore, Inc.
• Original Assignee: comScore, Inc.
• Inventors: Palit, Charles, Engel, Bill, Vinson, Michael, Goerlich, Bruce
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): Tran, Mai T
• Application Number:
  US14/031,548
• Publication Number:
  US 20140081703A1
• Time in Patent Office: 1,181 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 30/0202 Market predictions or forecasting for commercial activities
• CPC Class Codes: G06Q 30/0203 Market surveys; Market polls
• CPC Class Codes: G06Q 30/0207 Discounts or incentives, e.g. coupons or rebates

Abstract

A system and method that calibrates subject data for which a relationship to a target population is not known, so that the calibrated subject data can more accurately represent the target population. In many cases the calibration will involve the use of a differential weighting scheme applied to the data at the constituent level. The system and method allows the values of the observed variables in the subject data set to be weighted so that their incidence is equivalent to that of a reference population represented by a reference data set, even if the variables used in the reference data set to make estimates for the reference population were not collected or measured for the subject data set.

Claims

1. A method in a computing system for calibrating a subject data set of behavior data based on information from a reference data set of behavior data, each data set containing a plurality of participants and associated transactional data, the method comprising:

partitioning the reference data set into a plurality of reference data partitions, using a data partitioning scheme, no two reference data partitions sharing a participant in common;

partitioning the subject data set of behavior data into a plurality of subject data partitions using the data partitioning scheme, wherein;

each of the plurality of subject data partitions is based on a viewer characteristic that corresponds to a characteristic associated with a corresponding reference data partition; and

no two subject data partitions of the plurality of subject data partitions share a participant in common;

calculating weights associated with each of the plurality of subject data partitions to adjust a distribution of the plurality of subject data partitions based upon a distribution of the plurality of reference data partitions;

calculating a statistic for each of the plurality of subject data partitions; and

preparing adjusted calculated statistics by applying the calculated weight for each subject data partition to the calculated statistic for each subject data partition, the applied weights producing calibrated estimates of the statistics for the plurality of subject data partitions.

2. The method of claim 1, wherein each weight is determined by:

determining a reference portion by dividing the number of participants in a reference data partition by the total number of participants in the reference data set;

determining a subject portion by dividing the number of participants in a subject data partition by the total number of participants in the subject data set; and

dividing the first subject portion by the first reference portion.

3. The method of claim 1, wherein the weight is expressed as a percentage or as an absolute number.

4. The method of claim 1, further comprising preconditioning the reference data set to reduce bias or improve accuracy prior to partitioning the reference data set into a plurality of reference data partitions.

5. The method of claim 4, wherein the preconditioning is based on population census data and the preconditioning involves adjusting the reference data set so that it more closely resembles the plurality of participants represented by the population census data.

6. The method of claim 1, further comprising preconditioning the subject data set to reduce bias or improve accuracy prior to partitioning the subject data set into a plurality of subject data partitions.

7. The method of claim 1, further comprising generating an estimate for a variable contained in the reference data set that is not contained in the subject dataset by:

identifying a rate of occurrence of the variable in each of the plurality of reference data partitions; and

applying the identified rate of occurrence for the variable in each of the plurality of subject data partitions.

8. The method of claim 7, wherein the value of the variable for each subject data partition is expressed as a single value, an interval value, or a range of values for each subject data partition, each of the range of values having an associated probability.

9. The method of claim 1, wherein the transactional data represents financial transactions.

10. The method of claim 1, wherein the financial transactions represent transactions selected from the group consisting of transactions made with a credit card, online transactions, cash register transactions, frequent flier transactions, and loyalty program transactions.

11. The method of claim 1, wherein the transactional data represents television viewing behavior or web browsing behavior.

12. The method of claim 1, wherein the statistic is a count of the number of transactions or a count of the number of entities performing transactions.

13. The method of claim 1, further comprising:

combining the weights associated with each of the plurality of subject data partitions to calculate a total weight; and

assessing a degree of closeness between the subject data set and the reference data set based on the total weight.

14. The method of claim 1, wherein the behavior data comprises at least one of television viewing data, online video data, online audio data, or internet browsing data.

15. A non-transitory computer-readable medium encoded with instructions that, when executed by a processor, perform a method in a computing system for calibrating a subject data set of behavior data based on information from a reference data set of behavior data, each data set containing a plurality of participants and associated transactional data, the method comprising:

partitioning the reference data set into a plurality of reference data partitions, using a data partitioning scheme, no two reference data partitions sharing a participant in common;

partitioning the subject data set of behavior data into a plurality of subject data partitions using the data partitioning scheme, wherein;

each of the plurality of subject data partitions is based on a viewer characteristic that corresponds to a characteristic associated with a corresponding reference data partition; and

no two subject data partitions of the plurality of subject data partitions share a participant in common;

calculating weights associated with each of the plurality of subject data partitions to adjust a distribution of the plurality of subject data partitions based upon a distribution of the plurality of reference data partitions;

calculating a statistic for each of the plurality of subject data partitions; and

preparing adjusted calculated statistics by applying the calculated weight for each subject data partition to the calculated statistic for each subject data partition, the applied weights producing calibrated estimates of the statistics for the plurality of subject data partitions.

16. The non-transitory computer-readable medium of claim 15, wherein each weight is determined by:

determining a reference portion by dividing the number of participants in a reference data partition by the total number of participants in the reference data set;

determining a subject portion by dividing the number of participants in a subject data partition by the total number of participants in the subject data set; and

dividing the first subject portion by the first reference portion.

17. The non-transitory computer-readable medium of claim 15, wherein the weight is expressed as a percentage or as an absolute number.

18. The non-transitory computer-readable medium of claim 15, further comprising instructions that cause the computing system to generate an estimate for a variable contained in the reference data set that is not contained in the subject dataset by:

identifying a rate of occurrence of the variable in each of the plurality of reference data partitions; and

applying the identified rate of occurrence for the variable in each of the plurality of subject data partitions.

19. The non-transitory computer-readable medium of claim 18, wherein the value of the variable for each subject data partition is expressed as a single value, an interval value, or a range of values for each subject data partition, each of the range of values having an associated probability.

20. The non-transitory computer-readable medium of claim 15, wherein the transactional data represents financial transactions.

21. The non-transitory computer-readable medium of claim 15, wherein the financial transactions represent transactions selected from the group consisting of transactions made with a credit card, online transactions, cash register transactions, frequent flier transactions, and loyalty program transactions.

22. The non-transitory computer-readable medium of claim 15, wherein the transactional data represents television viewing behavior or web browsing behavior.

23. The non-transitory computer-readable medium of claim 15, wherein the behavior data comprises at least one of television viewing data, online video data, online audio data, or internet browsing data.

24. A method in a computing system for calibrating a subject data set of behavior data based on information from a reference data set of behavior data, each data set containing a plurality of participants, the method comprising:

partitioning the reference data set into a plurality of reference data partitions, using a data partitioning scheme;

partitioning the subject data set of behavior data into a plurality of subject data partitions using the data partitioning scheme, wherein;

each of the plurality of subject data partitions has one or more variables that are in common with the one or more variables associated with the corresponding reference data partition;

calculating weights associated with each of the plurality of subject data partitions to adjust a distribution of the plurality of subject data partitions based upon a distribution of the plurality of reference data partitions;

calculating a statistic for each of the plurality of subject data partitions; and

preparing adjusted calculated statistics by applying the calculated weight for each subject data partition to the calculated statistic for each subject data partition, the applied weights producing calibrated estimates of the statistics for the plurality of subject data partitions.

25. The method of claim 24, wherein each weight is determined by:

determining a reference portion by dividing the number of participants in a reference data partition by the total number of participants in the reference data set;

determining a subject portion by dividing the number of participants in a subject data partition by the total number of participants in the subject data set; and

dividing the first subject portion by the first reference portion.

26. The method of claim 24, further comprising preconditioning the subject data set to reduce bias or improve accuracy prior to partitioning the subject data set into a plurality of subject data partitions.

27. The method of claim 24, further comprising generating an estimate for a variable contained in the reference data set that is not contained in the subject dataset by:

identifying a rate of occurrence of the variable in each of the plurality of reference data partitions; and

applying the identified rate of occurrence for the variable in each of the plurality of subject data partitions.

28. The method of claim 27, wherein the value of the variable for each subject data partition is expressed as a single value, an interval value, or a range of values for each subject data partition, each of the range of values having an associated probability.

29. The method of claim 24, wherein the subject data set and the reference data set represent transactional data or behavioral data.

30. The method of claim 24, wherein the subject data set and the reference data set are associated with transactional data, and wherein the transactional data represent transactions selected from the group consisting of transactions made with a credit card, online transactions, cash register transactions, frequent flier transactions, and loyalty program transactions.

31. The method of claim 24, wherein the statistic is a count of the number of transactions or a count of the number of entities performing transactions.

32. The method of claim 24, further comprising:

combining the weights associated with each of the plurality of subject data partitions to calculate a total weight; and

assessing a degree of closeness between the subject data set and the reference data set based on the total weight.

33. The method of claim 24, wherein the behavior data comprises at least one of television viewing data, online video data, online audio data, or internet browsing data.