Complementary item recommendations using image feature data

• US 9,721,192 B2
• Filed: 03/09/2016
• Issued: 08/01/2017
• Est. Priority Date: 09/30/2011
• Status: Active Grant

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Bhardwaj, Anurag, Sundaresan, Neelakantan, Di, Wei, Jagadeesh, Vignesh, Piramuthu, Robinson
• Primary Examiner(s): Dang, Duy M
• Application Number:
  US15/065,122
• Publication Number:
  US 20160189011A1
• Time in Patent Office: 510 Days
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06Q 30/0629 for generating comparisons
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/41 based on statistical description of texture
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 10/56 relating to colour

Abstract

An apparatus and method to facilitate finding complementary recommendations are disclosed herein. One or more fashion trend or pleasing color combination rules are determined based on data obtained from one or more sources. One or more template images and rule triggers corresponding to the fashion trend or pleasing color combination rules are generated, each of the rule triggers associated with at least one of the template images. A processor compares a first image attribute of a particular one of the template images to a second image attribute of each of a plurality of inventory images corresponding to the plurality of inventory items to identify the inventory items complementary to the query image. The particular one of the template images is selected based on the rule trigger corresponding to the particular one of the template images being applicable for a query image.

Claims

1. A system comprising:

one or more processors; and

a memory storing instructions that, when executed by the processor, causes the one or more processors to perform operations comprising;

determining a rule that correlates a first image attribute associated with a first item to a second image attribute associated with a second item, the rule being determined based on data obtained from one or more sources;

based on the rule, generating an image index and a corresponding usage condition for the image index;

receiving a query image from a user interface;

determining the usage condition for the image index is satisfied by the query image based on a comparison of an image attribute of the image index to an image attribute of the query image;

in response to reception of the query image, selecting the image index based on the usage condition for the image index being satisfied by the query image;

identifying a set of item images among a plurality of item images stored in a database, the set of item image being identified based on the selected image index and the usage condition; and

causing presentation, at the user interface, of the identified set of item images.

2. The system of claim 1, wherein the operations further comprise:

wherein the causing of the presentation of the identified set of item images is performed as part of causing presentation of complementary images of the query image, the complementary images including the identified set of item images.

3. The system of claim 1, wherein:

the image attribute of the image index is selected, based on a confidence score that corresponds to at least one of a visual pattern, a dominant color of the template image, or a directionality of a template image associated with the image index, from a group consisting of a color histogram of the template image, the dominant color of the template image, and an orientation histogram of the template image.

4. The system of claim 1, wherein:

the data comprising of one or more trends specifying color combinations present in the identified one or more items.

5. The system of claim 1, wherein:

the data comprising of one or more trends specifying information obtained from a group consisting of users purchase behavior data, expert knowledge data, and social network data.

6. The system of claim 5, wherein:

the expert knowledge data includes an image that depicts a model wearing an outfit.

7. The system of claim 5 wherein:

the expert knowledge data include scientifically based color combination theory including a color scheme that indicate combinations of complementary colors or combinations of colors that create a color theme.

8. The system of claim 5, wherein:

the social network data includes color, or patterns obtained from an online social network associated with a user.

9. The system of claim 1, wherein:

the identified set of item images is included in a different category of items from a query item depicted in the query image.

10. A method comprising:

determining a rule that correlates a first image attribute associated with a first item to a second image attribute associated with a second item, the rule being determined based on data obtained from one or more sources;

based on the rule, generating an image index and a corresponding usage condition for the image index;

receiving a query image from a user interface;

determining the usage condition for the image index is satisfied by the query image based on a comparison of an image attribute of the image index to an image attribute of the query image;

in response to reception of the query image, selecting the image index based on the usage condition for the image index being satisfied by the query image;

identifying a set of item images among a plurality of item images stored in a database, the set of item image being identified based on the selected image index and the usage condition the identifying the set of item images being performed by a processor of a machine; and

causing presentation, at the user interface, of the identified set of item images.

11. The method of claim 10, further comprising:

wherein the causing of the presentation of the identified set of item images is performed as part of causing presentation of complementary images of the query image, the complementary images including the identified set of item images.

12. The method of claim 10, wherein:

the image attribute of the image index is selected, based on a confidence score that corresponds to at least one of a visual pattern, a dominant color of the template image, or a directionality of a template image associated with the image index, from a group consisting of a color histogram of the template image, the dominant color of the template image, and an orientation histogram of the template image.

13. The method of claim 10, wherein:

the data comprising of one or more trends specifying color combinations present in the identified one or more items.

14. The method of claim 10, wherein:

the data comprising of one or more trends specifying information obtained from a group consisting of users purchase behavior data, expert knowledge data, and social network data.

15. The method of claim 14, wherein:

the expert knowledge data includes at least one of color science information, a color chart, a complementary color scheme, or a color theme.

16. The method of claim 14, wherein:

the expert knowledge data includes an image that depicts a model wearing an outfit.

17. The method of claim 14, wherein:

the expert knowledge data include scientifically based color combination theory including a color scheme that indicate combinations of complementary colors or combinations of colors that create a color theme.

18. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

determining a rule that correlates a first image attribute associated with a first item to a second image attribute associated with a second item, the rule being determined based on data obtained from one or more sources;

based on the rule, generating an image index and a corresponding usage condition for the image index;

receiving a query image from a user interface;

determining the usage condition for the image index is satisfied by the query image based on a comparison of an image attribute of the image index to an image attribute of the query image;

in response to reception of the query image, selecting the image index based on the usage condition for the image index being satisfied by the query image;

identifying a set of item images among a plurality of item images stored in a database, the set of item image being identified based on the selected image index and the usage condition; and

causing presentation, at the user interface, of the identified set of item images.

19. The non-transitory machine-readable storage medium of claim 18, wherein the operations further comprise:

wherein the causing of the presentation of the identified set of item images is performed as part of causing presentation of complementary images of the query image, the complementary images including the identified set of item images.

20. The non-transitory machine-readable storage medium of claim 18, wherein:

the image attribute of the image index is selected, based on a confidence score that corresponds to at least one of a visual pattern, a dominant color of the template image, or a directionality of a template image associated with the image index, from a group consisting of a color histogram of the template image, the dominant color of the template image, and an orientation histogram of the template image.