Product Record Normalization System With Efficient And Scalable Methods For Discovering, Validating, And Using Schema Mappings

• US 20140358931A1
• Filed: 05/31/2013
• Published: 12/04/2014
• Est. Priority Date: 05/31/2013
• Status: Active Grant

• Current Assignee: Walmart Apollo, LLC (WalMart Inc.)
• Original Assignee: Wal-Mart Stores Texas LLC (WalMart Inc.)
• Inventors: Garera, Nikesh Lucky, Rampalli, Narasimhan, Ravikant, Dintyala Venkata Subrahmanya, Sun, Chong, Yalin, Heather Dawn, Subramaniam, Sirkanth
• Granted Patent: US 9,311,372 B2
• Time in Patent Office: Days
• US Class Current: 707/740
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database

Abstract

Systems and methods are disclosed herein for generating a normalized record from an import record, the normalized record having attribute-value pairs corresponding to a native schema. In import records, a plurality of attribute-value are identified each having an attribute label not found in a native schema. One or more attribute labels in the native schema having as possible values one or more values corresponding to the values of the plurality of attribute-value pairs are also identified. The computer system generates one or more normalization rules relating one or more attribute labels of the plurality of attribute-value pairs to at least a portion of the one or more attribute labels in the native schema. Normalization rules may be validated by crowdsourcing. Normalization rules may be applied by identifying implicated rules by classifying the import record and identifying rules applicable to the classification.

Claims

1. method for classification, the method comprising:

identifying, by the computer system, in one or more import records a plurality of attribute-value pairs each having an attribute label not found in a native schema and a value;

identifying, by the computer system, one or more attribute labels in the native schema having as possible values one or more values corresponding to the values of the plurality of attribute-value pairs; and

generating, by the computer system, one or more normalization rules relating one or more attribute labels of the plurality of attribute-value pairs to at least a portion of the one or more attribute labels in the native schema.

2. The method of claim 1, wherein the one or more import records are a plurality of import records;

and

wherein identifying in the one or more import records a plurality of attribute-value pairs further comprises identifying as the plurality of attribute-value pairs those attribute-value pairs in the plurality of import records having high frequency of usage relative to other attribute-value pairs in the plurality of import records having attribute labels not found in the native schema.

3. The method of claim 1, wherein the one or more import records are a plurality of import records;

and

wherein identifying in the one or more import records a plurality of attribute-value pairs further comprises;

classifying a first portion of the one or more import records as being of a first type according to a taxonomy;

classifying a second portion of the one or more import records as being of a second type according to the a taxonomy; and

identifying as the plurality of attribute-value pairs;

a first group of attribute-value pairs in the first portion having high frequency of usage relative to other attribute-value pairs in the first portion and not found in the native schema; and

a second group of attribute-value pairs in the second portion having high frequency of usage relative to other attribute-value pairs in the second portion and not found in the native schema.

4. The method of claim 3, wherein the generating the one or more normalization rules relating the one or more attribute labels of the plurality of attribute-value pairs to the at least the portion of the one or more attribute labels in the native schema further comprises:

generating one or more first rules for the first portion, the one or more first rules having the first type associated therewith;

generating one or more second rules for the second portion, the one or more second rules having the second type associated therewith.

5. The method of claim 4, further comprising:

receiving a non-normalized record;

classifying the non-normalized record as belonging to a selected type including one of the first type and the second type;

normalizing the non-normalized record according to one of the first and second rules corresponding to the selected type to generate a normalized record;

storing the normalized record in a database.

6. The method of claim 5, wherein the non-normalized record is a product record and the database is a product catalog.

7. The method of claim 1, further comprising:

normalizing a plurality of non-normalized records according to the one or more normalization rules to generate a plurality of provisionally normalized records;

transmitting the provisionally normalized records to a crowdsourcing forum;

receiving, from the crowdsourcing forum, one or more favorable validation decisions with respect to a first portion of the plurality of provisionally normalized records;

receiving, from the crowdsourcing forum, one or more unfavorable validation decisions with respect to a second portion of the plurality of provisionally normalized records;

identifying one or more first normalization rules from the one or more normalization rules, the first rules corresponding to the first portion of the plurality of provisionally normalized records; and

adding the one or more first normalization rules to a validated rule set.

8. The method of claim 7, further comprising:

identifying, one or more second normalization rules from the one or more normalization rules, the second rules corresponding to the second portion of the plurality of provisionally normalized records; and

discarding the one or more second normalization rules.

9. The method of claim 8, further comprising:

transmitting at least one of the second portion of the plurality of provisionally normalized records and the one or more second normalization rules to an analyst workstation network with an instruction to generate a valid normalization rule; and

receiving, from the analyst workstation network, at least one validated normalization rule.

10. A method for classification, the method comprising:

receiving, by a computer system, an import record;

identifying, by the computer system, one or more product types corresponding to the import record;

identifying, by the computer system, native attribute-value pairs in a plurality of native records belonging to the one or more product types;

identifying, by the computer system, from among a plurality of normalization rules one or more implicated normalization rules implicated by the identified native attribute-value pairs;

identifying, by the computer system, one or more non-native attribute labels in non-native attribute-value pairs in the import record;

applying, by the computer system, the one or more implicated normalization to the import record to generate normalized attribute-value pairs for the non-native attribute-value pairs; and

storing, by the computer system, a normalized version of the import record including the normalized attribute-value pairs.

11. The method of claim 10, wherein identifying the one or more product types comprises classifying the content record according to content thereof as corresponding to one or more nodes of a taxonomy representing the one or more product types.

12. The method of claim 11, wherein the taxonomy is a product catalog.

13. The method of claim 10, wherein the normalization rules take as input both at least one of the non-native attribute-value pairs and one or more items of contextual data for the at least one of the non-native attribute-value pairs.

14. A system for classification, the system comprising one or more processors and one or more memory devices operably coupled to the one or more memory devices, the one or more memory devices storing executable and operational data effective to cause the one or more processors to:

identify in one or more import records a plurality of attribute-value pairs each having an attribute label not found in a native schema and a value;

identify one or more attribute labels in the native schema having as possible values one or more values corresponding to the values of the plurality of attribute-value pairs; and

generate one or more normalization rules relating one or more attribute labels of the plurality of attribute-value pairs to at least a portion of the one or more attribute labels in the native schema.

15. The system of claim 14, wherein the one or more import records are a plurality of import records;

and

wherein the executable and operational data are further effective to cause the one or more processors to identify in the one or more import records a plurality of attribute-value pairs by identifying as the plurality of attribute-value pairs those attribute-value pairs in the plurality of import records having high frequency of usage relative to other attribute-value pairs in the plurality of import records having attribute labels not found in the native schema.

16. The system of claim 14, wherein the one or more import records are a plurality of import records;

and

wherein the executable and operational data are further effective to cause the one or more processors to identify in the one or more import records a plurality of attribute-value pairs by;

classifying a first portion of the one or more import records as being of a first type according to a taxonomy;

classifying a second portion of the one or more import records as being of a second type according to the a taxonomy; and

identifying as the plurality of attribute-value pairs;

a first group of attribute-value pairs in the first portion having high frequency of usage relative to other attribute-value pairs in the first portion and not found in the native schema; and

a second group of attribute-value pairs in the second portion having high frequency of usage relative to other attribute-value pairs in the second portion and not found in the native schema.

17. The system of claim 16, wherein the executable and operational data are further effective to cause the one or more processors to generate the one or more normalization rules relating the one or more attribute labels of the plurality of attribute-value pairs to the at least the portion of the one or more attribute labels in the native schema by:

generating one or more first rules for the first portion, the one or more first rules having the first type associated therewith;

generating one or more second rules for the second portion, the one or more second rules having the second type associated therewith.

18. The system of claim 17, wherein the executable and operational data are further effective to cause the one or more processors to:

receive a non-normalized record;

classify the non-normalized record as belonging to a selected type including one of the first type and the second type;

normalize the non-normalized record according to one of the first and second rules corresponding to the selected type to generate a normalized record;

store the normalized record in a database.

19. The system of claim 18, wherein the non-normalized record is a product record and the database is a product catalog.

20. The system of claim 14, wherein the executable and operational data are further effective to cause the one or more processors to:

normalize a plurality of non-normalized records according to the one or more normalization rules to generate a plurality of provisionally normalized records;

transmit the provisionally normalized records to a crowdsourcing forum;

receive, from the crowdsourcing forum, one or more favorable validation decisions with respect to a first portion of the plurality of provisionally normalized records;

receive, from the crowdsourcing forum, one or more unfavorable validation decisions with respect to a second portion of the plurality of provisionally normalized records;

identify one or more first normalization rules from the one or more normalization rules, the first rules corresponding to the first portion of the plurality of provisionally normalized records; and

add the one or more first normalization rules to a validated rule set.