DUPLICATE ITEM DETECTION SYSTEM AND METHOD

• US 20090089314A1
• Filed: 09/28/2007
• Published: 04/02/2009
• Est. Priority Date: 09/28/2007
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Hicks, Cory
• Granted Patent: US 7,827,186 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06Q 30/0603 Catalogue ordering

Abstract

A method of detecting contextual duplicate items can include identifying a plurality of representations of items in a data repository, each item representation including one or more textual attributes. A degree of fit between an item representation'"'"'s attributes and other items can be calculated. The degree of fit can reflect the relevance of the attributes of one item to the other item. A degree of association between the two item representations can be calculated based at least in part on the calculated degree of fit. The degree of association between the two item representations can reflect the similarity of the two items. The degree of association between the two item representations can be assessed to determine whether the items are contextual duplicates.

Claims

1. A computer-implemented method of detecting context-based duplicate items in an electronic catalog, the method comprising:

identifying a plurality of representations of items in a data repository of an electronic catalog from which to select items to recommend to a target user;

identifying one or more textual terms of each item representation, each textual term listed in a product description for a given item, the one or more textual terms describing the given item;

calculating degrees of fit between the textual terms of representations of first and second items selected from the plurality of item representations, the calculated degrees of fit reflecting the contextual similarities of the textual terms of the first and second item representations, wherein calculating degrees of fit comprises;

forming an initial matrix of values, each of the textual terms of the first and second item representations having a value represented in an initial matrix;

calculating a singular value decomposition of the initial matrix, the singular value decomposition comprising a left matrix, a singular value matrix, and a right transpose matrix;

reducing the dimension of one or more of the left, singular value, and right transpose matrices to create a reduced singular value decomposition; and

multiplying the matrices of the reduced singular value decomposition to create a reduced-dimension matrix approximating the initial matrix;

calculating a degree of similarity between the first and second item representations based at least in part on the calculated degrees of fit, the degree of similarity between the first and second item representations reflecting the similarity of the first and second item representations; and

assessing whether the first and second items are contextual duplicates based at least in part on the calculated degree of similarity.

2. The method of claim 1, wherein the representations comprise product descriptions.

3. The method of claim 1, wherein at least some of the items are apparel items.

4. The method of claim 1, further comprising filtering candidate recommendations based at least in part on determining that the first and second items are contextual duplicates.

5. A computer-implemented method of detecting context-based duplicate items, the method comprising:

identifying a plurality of representations of items in a data repository;

identifying one or more attributes of each item representation, each attribute comprising one or more textual terms;

calculating degrees of fit between the attributes of representations of first and second items selected from the plurality of item representations, the calculated degrees of fit reflecting the contextual similarities of the attributes of the first and second item representations;

calculating a degree of similarity between the first and second item representations based at least in part on the calculated degrees of fit, the degree of similarity between the first and second item representations reflecting the similarity of the first and second item representations; and

assessing whether the first and second items are contextual duplicates based at least in part on the calculated degree of similarity.

6. The method of claim 5, wherein calculating degrees of fit comprises:

forming an initial matrix of values corresponding to the attributes of the first and second item representations; and

using a singular value decomposition to reduce the dimension of the initial matrix to form a reduced-dimension matrix approximating the initial matrix.

7. The method of claim 5, further comprising removing one of the first and second item representations from an electronic catalog in response to determining that the first and second items are contextual duplicates.

8. The method of claim 5, further comprising filtering candidate recommendations based at least in part on determining that the first and second items are contextual duplicates.

9. The method of claim 5, wherein assessing whether the first and second items are contextual duplicates comprises comparing the calculated degree of similarity to a threshold.

10. The method of claim 5, further comprising providing one of the first and second item representations as a search result to a user in response to the other of the first and second items being provided as a search result to the user.

11. A computer system for detecting similarities between items represented in a data repository, the system comprising:

an item attributes analysis component configured to;

identify a plurality of representations of items in a data repository;

identify one or more attributes of each item representation, each attribute comprising one or more textual terms; and

calculate degrees of fit between the attributes of representations of first and second items selected from the plurality of item representations, the calculated degrees of fit reflecting the contextual similarities of the attributes of the first and second item representations; and

an association analysis component configured to;

calculate a degree of similarity between the first and second item representations based at least in part on the calculated degrees of fit, the degree of similarity between the first and second item representations reflecting the similarity of the first and second item representations; and

assess whether the first and second items are contextual duplicates based at least in part on the calculated degree of similarity.

12. The system of claim 11, wherein the textual terms are listed in a product description of each item representation.

13. The system of claim 11, wherein at least some of the items are apparel items.

14. The system of claim 11, wherein the association analysis component is further configured to calculate degrees of fit by forming an initial matrix of values corresponding to the attributes of the first and second item representations and by using a singular value decomposition to reduce the dimension of the initial matrix to form a reduced-dimension matrix approximating the initial matrix.

15. A computer-implemented method of detecting context-based similarities, the method comprising:

identifying a plurality of representations of items in a data repository;

identifying one or more attributes of each item representation, each attribute comprising one or more textual terms describing an item;

calculating degrees of fit between the attributes of representations of first and second items selected from the plurality of item representations, the calculated degrees of fit reflecting the contextual similarities of the attributes of the first and second item representations;

calculating a degree of similarity between the first and second item representations based at least in part on the calculated degrees of fit, the degree of similarity between the first and second item representations reflecting the similarity of the first and second items; and

storing the calculated degree of similarity in computer storage.

16. The method of claim 15, wherein at least some of the items are apparel items.

17. The method of claim 15, further comprising determining that the first and second items are contextual duplicates in response to determining that the calculated degree of fit exceeds a threshold.

18. The method of claim 17, further comprising removing one of the first and second item representations from an electronic catalog in response to determining that the first and second items are contextual duplicates.

19. The method of claim 17, further comprising filtering candidate recommendations based at least in part on determining that the first and second items are contextual duplicates.

20. The method of claim 17, further comprising providing one of the first and second item representations as a search result to a user in response to the other of the first and second item representations being provided as a search result to the user.

21. The method of claim 15, further comprising recommending one or more of the first and second item representations to a user based at least in part on the degree of similarity between the first and second items.

22. A computer-implemented method of assessing a degree of similarity between a first representation of an apparel item having a first textual attribute and a second representation of an apparel item having a second textual attribute, the method comprising:

calculating a first degree of contextual similarity between the first apparel item representation and the second textual attribute;

calculating a second degree of contextual similarity between the second apparel item representation and the first textual attribute; and

assessing a degree of similarity between the first and second apparel items based, at least in part, on the first and second calculated degrees of contextual similarity.

23. The method of claim 22, further comprising determining that the first and second apparel items are contextual duplicates in response to determining that the degree of contextual similarity between the first and second apparel items exceeds a threshold.

24. The method of claim 22, further comprising recommending one or more of the first and second apparel item representations to a user based at least in part on the degree of similarity between the first and second apparel items.

25. The method of claim 22, wherein calculating a first degree of contextual similarity comprises applying latent semantic analysis techniques to the first textual attribute and the second apparel item representation.