Complementary item recommendations using image feature data

• US 8,737,728 B2
• Filed: 09/28/2012
• Issued: 05/27/2014
• Est. Priority Date: 09/30/2011
• Status: Active Grant

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Bhardwaj, Anurag, Di, Wei, Jagadeesh, Vignesh, Piramuthu, Robinson, Sundaresan, Neelakantan
• Primary Examiner(s): Dang, Duy M
• Application Number:
  US13/631,839
• Publication Number:
  US 20130084000A1
• Time in Patent Office: 606 Days
• Field of Search: 382/162, 382/165, 382/167, 382/305, 705/26.1
• US Class Current: 382/165
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06Q 30/0629 for generating comparisons
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/41 based on statistical description of texture
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 10/56 relating to colour

Abstract

An apparatus and method to facilitate finding complementary recommendations are disclosed herein. One or more fashion trend or pleasing color combination rules are determined based on data obtained from one or more sources. One or more template images and rule triggers corresponding to the fashion trend or pleasing color combination rules are generated, each of the rule triggers associated with at least one of the template images. A processor compares a first image attribute of a particular one of the template images to a second image attribute of each of a plurality of inventory images corresponding to the plurality of inventory items to identify the inventory items complementary to the query image. The particular one of the template images is selected based on the rule trigger corresponding to the particular one of the template images being applicable for a query image.

Claims

1. A method to facilitate providing complementary recommendations, the method comprising:

determining one or more fashion trend or pleasing color combination rules based on data obtained from one or more sources;

generating one or more template images and rule triggers corresponding to the fashion trend or pleasing color combination rules, each of the rule triggers associated with at least one of the template images; and

comparing, using a processor, a first image attribute of a particular one of the template images to a second image attribute of each of a plurality of inventory images corresponding to the plurality of inventory items to identify the inventory items complementary to the query image, wherein the particular one of the template images is selected based on the rule trigger corresponding to the particular one of the template images being applicable for a query image, and wherein colors or patterns of each of the identified inventory items differ from the colors or patterns of the query image.

2. The method of claim 1, wherein the first image attribute is selected among a color histogram, a dominant color, or an orientation histogram corresponding to the particular one of the template images based on a confidence score associated with a visual pattern, the dominant color, or a directionality of the particular one of the template images.

3. The method of claim 1, wherein one or more of the identified inventory items is a different category of items from the query image.

4. The method of claim 3, wherein the category of items comprises clothing, shirts, skirts, pants, dresses, outerwear, shoes, handbags, accessories, bedding, upholstery, or furnishings.

5. The method of claim 1, wherein the data comprises one or more of purchasing behavior data, expert knowledge data, and social network data.

6. The method of claim 5, wherein the expert knowledge data includes an image of a celebrity or model wearing an outfit and accessories and the template image comprises the m age.

7. The method of claim 5, wherein the expert knowledge data includes color science-based information, color charts, or complementary color schemes and themes.

8. The method of claim 5, wherein the social network data includes images or non-pictorial information relating to fashion, color, or pattern from a user'"'"'s social circle, and the identified inventory items are presented to the user as personalized complementary recommendations.

9. A device, comprising:

at east one memory;

at least one processor in communication with the memory; and

one or more modules comprising instructions stored in the memory and executed by the processor to perform operations comprising;

determining one or more fashion trend or pleasing color combination rules based on data obtained from one or more sources;

generating one or more template images and rule triggers corresponding to the fashion trend or pleasing color combination rules, each of the rule triggers associated with at least one of the template images; and

comparing a first image attribute of a particular one of the template images to a second image attribute of each of a plurality of inventory images corresponding to the plurality of inventory items to identify the inventory items complementary to the query image, wherein the particular one of the template images is selected based on the rule trigger corresponding to the particular one of the template images being applicable for a query image, and wherein colors or patterns of each of the identified inventory items differ from the colors or patterns of the query image.

10. The device of claim 9, wherein the first image attribute is selected among a color histogram, a dominant color, or an orientation histogram corresponding to the particular one of the template images based on a confidence score associated with a visual pattern, the dominant color, or a directionality of the particular one of the template images.

11. The device of claim 9, wherein one or more of the identified inventory items is a different category of items from the query image.

12. The device of claim 11, wherein the category of items comprises clothing, shirts, skirts, pants, dresses, outerwear, shoes, handbags, accessories, bedding, upholster, or furnishings.

13. The device of claim 9, wherein the data comprises one or more of purchasing behavior data, expert knowledge data, and social network data.

14. The device of claim 13, wherein the expert knowledge data includes an image of a celebrity or model wearing an outfit and accessories and the template image comprises the image.

15. The device of claim 13, wherein the expert knowledge data includes color science-based information, color charts, or complementary color schemes and themes.

16. The device of claim 13, wherein the social network data includes images or non pictorial information relating to fashion, color, or pattern from a user'"'"'s social circle, and the identified inventory items are presented to the user as personalized complementary recommendations.

17. A non-transitory computer readable medium including instructions, when executed by a processor, causes the processor to perform operations comprising:

determining one or more fashion trend or pleasing color combination rules based on data obtained from one or more sources;

generating one or more template images and rule triggers corresponding to the fashion trend or pleasing color combination rules, each of the rule triggers associated with at least one of the template images; and

comparing a first image attribute of a particular one of the template images to a second image attribute of each of a plurality of inventory images corresponding to the plurality of inventory items to identify the inventory items complementary to the query image, wherein the particular one of the template images is selected based on the rule trigger corresponding to the particular one of the template images being applicable for a query image, and wherein colors or patterns of each of the identified inventory items differ from the colors or patterns of the query image.

18. The non-transitory computer readable medium of claim 17, wherein one or more of the identified inventory items is a different category of items from the query image.

19. The non-transitory computer readable medium of claim 17, wherein the data comprises one or more of purchasing behavior data, expert knowledge data, and social network data.

20. The method of claim 19, wherein the expert knowledge data includes an image of a celebrity or model wearing an outfit and accessories and the template image comprises the image, wherein the expert knowledge data includes color science-based information, color charts, or complementary color schemes and themes, and wherein the social network data includes images or non-pictorial information relating to fashion, color, or pattern from a user'"'"'s social circle and the identified inventory items comprise personalized complementary recommendations for the user.