Complementary item recommendations using image feature data

• US 9,311,666 B2
• Filed: 03/18/2014
• Issued: 04/12/2016
• Est. Priority Date: 09/30/2011
• Status: Expired due to Fees

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Bhardwaj, Anurag, Di, Wei, Jagadeesh, Vignesh, Piramuthu, Robinson, Sundaresan, Neelakantan
• Primary Examiner(s): Dang, Duy M
• Application Number:
  US14/218,772
• Publication Number:
  US 20140201028A1
• Time in Patent Office: 756 Days
• Field of Search: 382/165, 703/26.1, 703/26.7
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06Q 30/0629 for generating comparisons
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/41 based on statistical description of texture
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 10/56 relating to colour

Abstract

An apparatus and method to facilitate finding complementary recommendations are disclosed herein. One or more fashion trend or pleasing color combination rules are determined based on data obtained from one or more sources. One or more template images and rule triggers corresponding to the fashion trend or pleasing color combination rules are generated, each of the rule triggers associated with at least one of the template images. A processor compares a first image attribute of a particular one of the template images to a second image attribute of each of a plurality of inventory images corresponding to the plurality of inventory items to identify the inventory items complementary to the query image. The particular one of the template images is selected based on the rule trigger corresponding to the particular one of the template images being applicable for a query image.

Claims

1. A method comprising:

determining a rule based on data obtained from one or more sources;

based on the rule, generating a template image and a corresponding rule trigger that defines a usage condition for the template image; and

identifying an item depicted in an item image based on a query image satisfying the usage condition defined by the rule trigger that corresponds to the template image, the template image being a basis for identifying the item image, the identifying of the item being performed by a processor of a machine based on a comparison of an image attribute of the template image to an image attribute of the item image.

2. The method of claim 1, wherein:

the image attribute of the template image is selected, based on a confidence score that corresponds to at least one of a visual pattern, a dominant color of the template image, or a directionality of the template image, from a group consisting of a color histogram of the template image, the dominant color of the template image, and an orientation histogram of the template image.

3. The method of claim 1, wherein:

the identified item is included in a different category of items from a query item depicted in the query image.

4. The method of claim 3, wherein:

the different category of items is at least one of clothing, textiles, bedding, shoes, bags, upholstery, electronics, home and garden, or collectibles.

5. The method of claim 1, wherein:

the data includes at least one of purchase behavior data, expert knowledge data, or social network data.

6. The method of claim 5, wherein:

the expert knowledge data includes a celebrity image that depicts a celebrity wearing an outfit; and

the template image includes the celebrity image.

7. The method of claim 5, wherein:

the expert knowledge data includes at least one of color science information, a color chart, a complementary color scheme, or a color theme.

8. The method of claim 5, wherein:

the social network data includes information relating to fashion, color, or patterns from a social circle of a user; and

the method further comprises

presenting the identified item to the user as a recommendation.

9. A device comprising:

a memory;

one or more processors communicatively coupled to the memory; and

one or more modules that configure the one or more processors to perform operations comprising;

determining a rule based on data obtained from one or more sources;

based on the rule, generating a template image and a corresponding rule trigger that defines a usage condition for the template image; and

identifying an item depicted in an item image based on a query image satisfying the usage condition defined by the rule trigger that corresponds to the template image, the template image being a basis for identifying the item image, the identifying of the item being based on a comparison of an image attribute of the template image to an image attribute of the item image.

10. The device of claim 9, wherein:

the image attribute of the template image is selected, based on a confidence score that corresponds to at least one of a visual pattern, a dominant color of the template image, or a directionality of the template image, from a group consisting of a color histogram of the template image, the dominant color of the template image, and an orientation histogram of the template image.

11. The device of claim 9, wherein:

the identified item is included in a different category of items from a query item depicted in the query image.

12. The device of claim 11, wherein:

the different category of items is at least one of clothing, textiles, bedding, shoes, bags, upholstery, electronics, home and garden, or collectibles.

13. The device of claim 9, wherein:

the data includes at least one of purchasing behavior data, expert knowledge data, or social network data.

14. The device of claim 13, wherein:

the expert knowledge data includes a celebrity image that depicts a celebrity wearing an outfit; and

the template image includes the celebrity image.

15. The device of claim 13, wherein:

the expert knowledge data includes at least one of color science information, a color chart, a complementary color scheme, or a color theme.

16. The device of claim 13, wherein:

the social network data includes information relating to fashion, color, or patterns from a social circle of a user; and

the operations further comprise

presenting the identified item to the user as a recommendation.

17. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

determining a rule based on data obtained from one or more sources;

based on the rule, generating a template image and a corresponding rule trigger that defines a usage condition for the template image; and

identifying an item depicted in an item image based on a query image satisfying the usage condition defined by the rule trigger that corresponds to the template image, the template image being a basis for identifying the item image, the identifying of the item being performed by the one or more processors of the machine based on a comparison of an image attribute of the template image to an image attribute of the item image.

18. The non-transitory machine-readable storage medium of claim 17, wherein:

the identified item is included in a different category of items from a query item depicted in the query image.

19. The non-transitory machine-readable storage medium of claim 17, wherein:

the data includes at least one of purchasing behavior data, expert knowledge data, or social network data.

20. The non-transitory machine-readable storage medium of claim 19, wherein:

the expert knowledge data includes a celebrity image that depicts a celebrity wearing an outfit;

the template image includes the celebrity image;

the expert knowledge data includes at least one of color science information, a color chart, a complementary color scheme, or a color theme;

the social network data includes information relating to fashion, color, or patterns from a social circle of a user; and

the operations further comprise

presenting the identified item to the user as a recommendation.