IDENTIFYING PRODUCT REFERENCES IN USER-GENERATED CONTENT

• US 20140149105A1
• Filed: 11/28/2012
• Published: 05/29/2014
• Est. Priority Date: 11/28/2012
• Status: Active Grant

• Current Assignee: Walmart Apollo, LLC (WalMart Inc.)
• Original Assignee: Wal-Mart Stores Texas LLC (WalMart Inc.)
• Inventors: Lamba, Digvijay Singh, Chai, Xiaoyong, Whisler, Nicole
• Granted Patent: US 9,256,593 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates

Abstract

Systems and methods are disclosed herein for extracting products referenced in a document. A document is analyzed to identify a product type that is referenced in the document. Attributes are extracted from the document. A set of candidate products are identified corresponding to the extracted attributes. A score is calculated for the candidate products and the products are further selected or filtered based on the score, whitelist rules, and blacklist rules in order to identify one or more inferred products referenced by the document. The whitelist and blacklist rules may take as inputs a domain, a user identifier, and keywords included in the document. A set of sufficient attributes may be identified for each product type. Selection of a candidate product may be based at least in part on the document including all of the attributes in the set of sufficient attributes.

Claims

1. A method for product extraction, the method comprising:

receiving, by a computer system, a document;

identifying, by the computer system, a product type for the document according to content of the document;

extracting, by the computer system, product attributes and attribute values from the document;

comparing, by the computer system, the extracted attributes to a sufficient attribute set specific to the identified product type; and

selecting, by the computer system, an inferred product based at least on part on the product having the extracted attribute values for the extracted attributes belonging to the sufficient attribute set.

2. The method of claim 1, wherein selecting the product based at least on part on the product having the extracted attribute values for the extracted attributes belonging to the sufficient attribute set further comprises:

identifying a plurality of candidate products;

calculating a score for each candidate product of the plurality of candidate products; and

selecting the inferred product based at least in part on the calculated scores for the plurality of candidate products.

3. The method of claim 2, wherein selecting the inferred product based at least in part on the calculated scores for the plurality of candidate products further comprises at least one of:

removing candidate products from the plurality of candidate products if application of a blacklist rule to the document so indicates; and

selecting the inferred product as specified by a whitelist rule if application of the whitelist rule to the document so indicates.

4. The method of claim 3, wherein at least one of the blacklist rule and whitelist rule take as an input a list of keywords from the document.

5. The method of claim 3, wherein at least one of the blacklist rule and whitelist rule take as an input an author of the document.

6. The method of claim 3, wherein at least one of the blacklist rule and whitelist rule take as an input a web domain of the document.

7. The method of claim 3, wherein at least one of the blacklist rule and whitelist rule take as an input a uniform resource locator for the document.

8. The method of claim 2, wherein calculating the score for each candidate product of the plurality of candidate products further comprises calculating a score based on at least one of:

a proximity of context of the document to a context associated with the candidate product;

a similarity of attributes of the candidate product and the extracted attributes;

textual properties of the extracted attributes; and

a number of actual mentions of the candidate product and any synonyms thereof in the document.

9. The method of claim 8, wherein the textual properties of the extracted attributes include a user of capitalization and the presence of digits in the extracted attributes.

10. The method of claim 2, wherein selecting the inferred product based at least in part on the calculated scores for the plurality of candidate products further comprises:

removing candidate products from the plurality of candidate products based on comparison of the calculated scored thereof to a threshold;

removing candidate products from the plurality of candidate products based on dissimilarity of product domains of the removed candidate products to other candidate products of the plurality of candidate products; and

identify a highest scoring product of the plurality of candidate products remaining

11. A system for product extraction, the system comprising one or more processors and one or more memory devices operably coupled to the one or more processors, the one or more memory devices storing executable and operational data effective to cause the one or more processors to:

receive a document;

identify a product type for the document according to content of the document;

extract product attributes and attribute values from the document;

compare the extracted attributes to a sufficient attribute set specific to the identified product type; and

select an inferred product based at least on part on the product having the extracted attribute values for the extracted attributes belonging to the sufficient attribute set.

12. The system of claim 11, wherein the executable and operational data are further effective to cause the one or more processors to select the product based at least on part on the product having the extracted attribute values for the extracted attributes belonging to the sufficient attribute set by:

identifying a plurality of candidate products;

calculating a score for each candidate product of the plurality of candidate products; and

selecting the inferred product based at least in part on the calculated scores for the plurality of candidate products.

13. The system of claim 12, wherein the executable and operational data are further effective to cause the one or more processors to select the inferred product based at least in part on the calculated scores for the plurality of candidate products by at least one of:

removing candidate products from the plurality of candidate products if application of a blacklist rule to the document so indicates; and

selecting the inferred product as specified by a whitelist rule if application of the whitelist rule to the document so indicates.

14. The system of claim 13, wherein at least one of the blacklist rule and whitelist rule take as an input a list of keywords from the document.

15. The system of claim 13, wherein at least one of the blacklist rule and whitelist rule take as an input an author of the document.

16. The system of claim 13, wherein at least one of the blacklist rule and whitelist rule take as an input a web domain associated with the document.

17. The system of claim 13, wherein at least one of the blacklist rule and whitelist rule take as an input a uniform resource locator for the document.

18. The system of claim 12, wherein the executable and operational data are further effective to cause the one or more processors to calculate the score for each candidate product of the plurality of candidate products by calculating a score based on at least one of:

a proximity of context of the document to a context associated with the candidate product;

a similarity of attributes of the candidate product and the extracted attributes;

textual properties of the extracted attributes; and

a number of actual mentions of the candidate product and any synonyms thereof in the document.

19. The system of claim 18, wherein the textual properties of the extracted attributes include a user of capitalization and the presence of digits in the extracted attributes.

20. The system of claim 12, wherein the executable and operational data are further effective to cause the one or more processors to select the inferred product based at least in part on the calculated scores for the plurality of candidate products by:

removing candidate products from the plurality of candidate products based on comparison of the calculated scored thereof to a threshold;

removing candidate products from the plurality of candidate products based on dissimilarity of product domains of the removed candidate products to other candidate products of the plurality of candidate products; and

identify a highest scoring product of the plurality of candidate products remaining