System and method for analyzing and correcting retail data

• US 20050187972A1
• Filed: 02/20/2004
• Published: 08/25/2005
• Est. Priority Date: 02/20/2004
• Status: Active Grant

• Current Assignee: Information Resources Incorporated
• Original Assignee: Symphonyiri Group Incorporated (New Mountain Capital Group LLC)
• Inventors: Kruger, Michael W., Johnson, Arvid C., Bergeon, Cheryl G.
• Granted Patent: US 7,873,529 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06Q 10/063 Operations research, analysis or management
• CPC Class Codes: G06Q 10/0637 Strategic management or analysis, e.g. setting a goal or target of an organisation; Planning actions based on goals; Analysis or evaluation of effectiveness of goals
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 30/0204 Market segmentation

Abstract

A computer system and method is disclosed that analyzes and corrects retail data. The system and method includes several client workstations and one or more servers coupled together over a network. A database stores various data used by the system. A business logic server uses competitive and complementary fusion to analyze and correct some of the data sources stored in database server. The data fusion process itself is an iterative one—utilizing both competitive and complementary fusion methods. In competitive fusion, two or more data sources that provide overlapping attributes are compared against each other. More accurate/reliable sources are used to correct less accurate/reliable sources. In complementary fusion, relationships modeled where data sources overlap are projected to areas of the data framework in which fewer sources exist—enhancing the accuracy/reliability of those fewer sources even in the absence of the other sources upon which the models were based.

Claims

1. A method comprising:

identifying a plurality of data sources, wherein at least a first data source is more accurate than a second data source;

identifying a plurality of overlapping attribute segments to use for comparing the data sources;

calculating a factor as a function of each of the plurality of overlapping attribute segments; and

using the factors to update a first group of values in the second data source to reduce bias.

2. The method of claim 1, further comprising:

using the factors to update a second group of values in the second data source to reduce incompleteness.

3. The method of claim 2, wherein the first and second groups of values are the same group.

4. The method of claim 2, wherein the first and second groups of values have at least some overlap.

5. The method of claim 2, wherein the first data source is point-of-sale data and the second data source is consumer panel data.

6. The method of claim 2, wherein the first and second sets of values are based on volume.

7. The method of claim 2, wherein the first and second sets of values are based on dollars.

8. The method of claim 1, wherein the first data source is point-of-sale data and the second data source is consumer panel data.

9. The method of claim 1, wherein the first data source is manufacturer-supplied shipment data.

10. The method of claim 1, wherein the first data source is retailer supplied data.

11. The method of claim 1, wherein the second data source is survey-based data.

12. The method of claim 11, wherein the second data source includes causal or promotional data.

13. The method of claim 1, wherein the second data source is sample-based data.

14. The method of claim 13, wherein the second data source includes causal or promotional data.

15. The method of claim 1, wherein the first and second sets of values are based on volume.

16. The method of claim 1, wherein the first and second sets of values are based on dollars.

17. The method of claim 1, wherein the data sources are compared based on a common time dimension.

18. The method of claim 1, wherein the data sources are compared based on a common venue dimension.

19. The method of claim 1, wherein the data sources are compared based on a common product dimension.

20. A method comprising:

receiving point-of-sale data and panel data on a periodic basis;

identifying a plurality of product identifiers aid a plurality of attributes to analyze;

retrieving and summarizing the point-of-sale data and the panel data by the plurality of product identifiers, the plurality of attributes, and a plurality of corresponding attribute segments for a specified time period;

calculating a factor for each attribute segment of a particular attribute; and

applying the factors for the particular attribute segment to the panel data to correct panel bias.

21. The method of claim 20, further comprising:

repeating the calculating and applying steps for each attribute segment of the plurality of attributes.

22. The method of claim 21, wherein the factor for each attribute segment is calculated by dividing a point-of-sale volume amount by a corresponding panel volume amount.

23. The method of claim 21, wherein the factor for each attribute segment is applied to the panel data by multiplying each of a plurality of panel data volume amounts by the factor for the corresponding attribute segment.

24. The method of claim 21, further comprising:

prior to said retrieving and summarizing step, preparing a table mapping the product identifiers to the corresponding attribute segments.

25. The method of claim 21, wherein the factors are only calculated for each attribute that is determined to be relevant.

26. The method of claim 21, wherein the factors are only calculated for each attribute segment that is determined to be significant.

27. The method of claim 21, further comprising:

after calculating each attribute segment factor, saving the attribute segment factor in a database.

28. The method of claim 21, further comprising:

applying at least one of the factors to the panel data to correct incompleteness.

29. The method of claim 21, wherein when at least two measures are available for a same factor, a blended factor is calculated and is used to reduce bias in the panel data.

30. The method of claim 29, wherein when calculating the blended factor, a more accurate data source is given a higher relative weight and a less accurate data source is given a lower relative weight.

31. The method of claim 21, wherein the product identifiers are uniform product codes.

32. The method of claim 21, wherein the product identifiers are SKU numbers.

33. A method comprising:

receiving point-of-sale data and panel data on a periodic basis;

identifying a plurality of product identifiers and a plurality of attributes to analyze;

retrieving and summarizing the point-of-sale data and the panel data by the plurality of product identifiers, the plurality of attributes, and a plurality of corresponding attribute segments for a specified the period;

calculating a factor for each attribute segment of the plurality of attributes; and

applying the factors for each particular attribute segment to the panel data to correct panel bias.

34. The method of claim 33, wherein the factor for each attribute segment is calculated by dividing a point-of-sale volume amount by a corresponding panel volume amount.

35. The method of claim 33, wherein attribute segment factors are applied to the panel data by multiplying each of a plurality of panel data volume amounts by the factors appropriate for the corresponding attribute segment.

36. The method of claim 33, wherein the factors are only calculated for each attribute that is determined to be relevant.

37. The method of claim 33, wherein the factors are only calculated for each attribute segment that is determined to be significant.

38. The method of claim 33, further comprising:

applying at least one of the factors to the panel data to correct incompleteness.

40. The method of claim 33, wherein the product identifiers are SKU numbers.

39. The method of claim 331 wherein the product identifiers are uniform product codes.

41. A method, comprising:

identifying a plurality of product identifiers and a plurality of attributes to analyze for at least two data sources, wherein at least a first data source is more accurate than a second data source;

retrieving and summarizing the first data source and the second data source by the plurality of product identifiers, the plurality of attributes, and a plurality of corresponding attribute segments for a specified time period;

calculating a plurality of factors, wherein one factor is calculated for each attribute segment of the plurality of attributes;

applying the factors to the second data source to reduce bias; and

applying the factors to the second data source to reduce incompleteness.

43. The method of claim 41, wherein the factor applying steps are repeated multiple times to further correct the second data source.

44. The method of claim 41, wherein the first data source is point-of-sale data and the second data source is consumer panel data.

45. The method of claim 41, wherein the product identifiers are uniform product codes.

46. The method of claim 41, wherein the product identifiers are SKU numbers.

42. The method of 41, further comprising:

determining that additional data sources are available for comparison;

using the additional data sources to calculate additional factors; and

applying the additional factors to the second data source.

47. A system comprising:

one or more servers being operable to store retail data from at least two data sources, store product identifier and attribute categorizations, and store a plurality of factor calculations;

wherein the at least two data sources includes a first data source that is more accurate than a second data source; and

wherein one or more of said servers contains business logic that is operable to identify and retrieve a plurality of overlapping attribute segments to use for comparing the at least two data sources, compare each of the overlapping attribute segments, calculate a factor for each of the overlapping attribute segments, and use the factors to update a first group of values in the second data source to reduce bias.

48. The system of claim 47, wherein the one or more servers are further operable to use the factors to update a second group of values in the second data source to reduce incompleteness.

49. The system of claim 47, wherein the one or more servers are further operable to calculate the factor for each overlapping attribute segment by dividing a first data source volume amount by a corresponding second data source volume amount.

50. The system of claim 47, wherein the one or more servers are further operable apply the factor for each attribute segment to the second data source by multiplying each of a plurality of second data source volume amounts by the factor for the corresponding attribute segment.

51. The system of claim 47, wherein the one or more servers are further operable to only calculate the factors for each attribute that is determined to be relevant.

52. The system of claim 47, wherein the one or more servers are further operable to only calculate the factors for each attribute segment that is determined to be significant.

53. The system of claim 47, wherein the one or more servers are further operable to save each attribute segment factor in a database.

54. The system of claim 47, wherein the one or more servers are further operable to apply at least one of the factors to the second data source to correct incompleteness.

55. The system of claim 54, wherein the one or more servers are further operable to calculate a blended factor when at least two measures are available for a same factor, said blended factor being used to reduce bias in the second data source.

56. The system of claim 55, wherein the one or more servers are further operable to calculate the blended factor by giving the more accurate data source a higher relative weight and by giving the less accurate data source a lower relative weight.

57. The system of claim 47, wherein the one or more servers are further operable to calculate a blended factor when at least two measures are available for a same factor, said blended factor being used to reduce bias in the second data source.

58. The system of claim 57, wherein the one or more servers are further operable to calculate the blended factor by giving the more accurate data source a higher relative weight and by giving the less accurate data source a lower relative weight.

59. The system of claim 47, further comprising:

a number of user stations coupled to the one or more servers over a network, wherein each of the user stations includes a respective one of a number of displays, and is operable to present a graphical user interface that allows a user to administer a plurality of settings used for analyzing and correcting the data sources.

60. The system of claim 47, further comprising:

a number of user stations coupled to the one or more servers over a network, wherein each of the user stations includes a respective one of a number of displays, and is operable to present a graphical user interface that allows a user to view the corrected data in a multi-dimensional format.

61. An apparatus, comprising:

a device encoded with logic executable by one or more processors to;

identify and retrieve a plurality of overlapping attribute segments to use for comparing at least two data sources, wherein the at least two data sources includes a first data source that is more accurate than a second data source, compare each of the overlapping attribute segments, calculate a factor for each of the overlapping attribute segments, and use the factors to update a first group of values in the second data source to reduce bias.

62. The apparatus of claim 61, wherein the device has further logic that is operable to apply all of the factors to the second data source to correct incompleteness.

63. The apparatus of claim 61, wherein the device includes a removable memory device carrying a number of processor executable instructions to define the logic.

64. The apparatus of claim 61, wherein the removable memory device includes a disk.

65. The apparatus of claim 61, wherein the device is in the form of one or more parts of a computer network carrying one or more signals encoding the logic.

66. A method comprising:

identifying a plurality of data sources in at least one computer database, wherein at least a first data source is more accurate than a second data source;

identifying a plurality of overlapping attribute segments to use for comparing the data sources;

calculating at least one factor as a function of at least one of the overlapping attribute segments; and

using the at least one factor to create modified values of a first group of values in the second data source, said modified values being more accurate than the first group of values.

67. The method of claim 66, wherein the first data source is point-of-sale data and the second data source is consumer panel data.

68. The method of claim 66, wherein the first data source does not include shipment data.

69. The method of claim 68, wherein the second data source does not include shipment data.

70. The method of claim 66, wherein the data sources are compared based on a common time dimension.

71. The method of claim 66, wherein the data sources are compared based on a common venue dimension.

72. The method of claim 66, wherein the data sources are compared based on a common product dimension.

73. The method of claim 66, further comprising:

determining that additional data sources are available for comparison;

using the additional data sources to calculate additional factors; and

applying the additional factors to the second data source.

74. The method of claim 66, wherein the using the at least one factor step is repeated multiple times to further improve the accuracy of the second data source.

75. The method of claim 66, wherein the first data source is point-of-sale data that was captured with a bar code reader.

76. A system comprising:

at least one computer being operable to store retail data from at least two data sources, store product identifier and attribute categorizations, and store a plurality of factor calculations;

wherein the at least two data sources includes a first data source that is more accurate than a second data source; and

wherein the at least one computer contains business logic that is operable to identify and retrieve a plurality of overlapping attribute segments to use for comparing the at least two data sources, compare selected overlapping attribute segments, calculate at least one factor for at least one of the overlapping attribute segments, and use the at least one factor to create modified values of a first group of values in the second data source, said modified values being more accurate than the first group of values.

77. The system of claim 76, wherein the at least one computer is further operable to calculate the at least one factor by dividing a first data source volume amount by a corresponding second data source volume amount.

78. The system of claim 76, wherein the at least one computer is further operable to apply the at least one factor to the second data source by multiplying each of a plurality of second data source volume amounts by the factor for the at least one corresponding attribute segment.

79. The system of claim 76, wherein the at least one computer is further operable to calculate a blended factor when at least two measures are available for a same factor, said blended factor being used to improve the accuracy of the second data source.

80. An apparatus, comprising:

a device encoded with logic executable by one or more processors to;

identify and retrieve a plurality of overlapping attribute segments to use for comparing at least two data sources, wherein the at least two data sources includes a first data source that is more accurate than a second data source, compare selected overlapping attribute segments, calculate at least one factor for at least one of the overlapping attribute segments, and use the at least one factor to create modified values of a first group of values in the second data source, said modified values being more accurate than the first group of values.