Item recommendations using image feature data

• US 8,908,962 B2
• Filed: 09/28/2012
• Issued: 12/09/2014
• Est. Priority Date: 09/30/2011
• Status: Active Grant

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Bhardwaj, Anurag, Di, Wei, Hamid, Muhammad Raffay, Piramuthu, Robinson, Sundaresan, Neelakantan
• Primary Examiner(s): Dang, Duy M
• Application Number:
  US13/631,842
• Publication Number:
  US 20130084001A1
• Time in Patent Office: 802 Days
• Field of Search: 382/162, 382/165, 382/167, 382/305, 705/26.1
• US Class Current: 382/165
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06Q 30/0629 for generating comparisons
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/41 based on statistical description of texture
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 10/56 relating to colour

Abstract

An apparatus and method to facilitate finding recommendations for a query image are disclosed herein. A color histogram is determined corresponding to the query image. Determining at least one of a visual pattern included in the query image, a dominant color of the query image, or an orientation histogram corresponding to the query image. Performing comparison of a first image attribute of the query image to a second image attribute of an inventory image corresponding to an inventory item, wherein the first image attribute used in the comparison is selected from among the color histogram, the dominant color, and the orientation histogram. The selection of the first image attribute is based on a confidence score associated with the visual pattern, the dominant color, or a directionality present in the query image.

Claims

1. Method to facilitate providing recommendations, the method comprising:

determining, using a processor, a color histogram corresponding to a query image;

determining at least one of a visual pattern included in the query image, a dominant color of the query image, or an orientation histogram corresponding to the query image; and

comparing a first image attribute of the query image to a second image attribute of an inventory image corresponding to an inventory item, the first image attribute used in the comparison selected from among the color histogram, the dominant color, and the orientation histogram, and

wherein the selection of the first image attribute is based on a confidence score associated with the visual pattern, the dominant color, or a directionality present in the query image.

2. The method of claim 1, wherein the color histogram is in accordance with a hue, saturation, and value (HSV) color space.

3. The method of claim 1, wherein the visual pattern comprises polka dots, stripes, check, aid, floral, or other repeating pattern.

4. The method of claim 1, wherein the comparison of the first image attribute to the second image attribute comprises comparing the color histogram of the query image to a color histogram of the inventory image, if the confidence score of the determined visual pattern is above a pre-defined threshold.

5. The method of claim 1, wherein the comparison of the first image attribute to the second image attribute comprises comparing the color histogram of the query image to a color histogram of the inventory image, if no visual pattern is detected and the confidence score of the determined dominant color is below a pre-defined threshold.

6. The method of claim 1, wherein the comparison of the first image attribute to the second image attribute comprises comparing the dominant color of the query image to a dominant color of the inventory image, if no visual pattern is detected and the confidence score of the determined dominant color is above a pre-defined threshold.

7. The method of claim 1, wherein the directionality is detected from the orientation histogram.

8. The method of claim 7, wherein the comparison of the first image attribute to the second image attribute comprises comparing the color histogram of the query image to a color histogram of the inventory image, if no visual pattern is detected and the confidence score of the directionality is below a pre-defined threshold.

9. The method of claim 7, wherein the comparison of the first image attribute to t second image attribute comprises comparing the color histogram of the query image to a color histogram of the inventory image and comparing the orientation histogram of the query to an orientation histogram of the inventory image, if no visual pattern is detected and the confidence score of the directionality is above a pre-defined threshold.

10. The method of claim 1, further comprising designating a sampling area of the query image and using the sampling area to determine the color histogram and at least one of the visual pattern, the dominant color, or the orientation histogram, wherein the sampling area comprises a central portion of the query image.

11. The method of claim 1, wherein the comparison of the first image attribute to the second image attribute comprises calculating a similarity score for each pair of the inventory image from a plurality of inventory images corresponding to a plurality of inventory items and the query image.

12. The method of claim 11, further comprising providing recommendations of inventory items corresponding to the query image to a remote device, the recommendations comprising a list of the inventory items ordered by the similarity scores of respective pairs of t inventory image and the query image.

13. A device, comprising:

at least one memory;

at least one processor in communication with the memory; and

one or more modules comprising instructions stored in the memory and executed by the processor to perform operations comprising;

determining a color histogram corresponding to a query image;

determining at least one of a visual pattern included in the query image, a dominant color of the query image, or an orientation histogram corresponding to the query image; and

comparing a first image attribute of the query image to a second image attribute of an inventory image corresponding to an inventory item, the first image attribute used in the comparison selected from among the color histogram, the dominant color, and the orientation histogram, and

wherein the selection of the first image attribute is based on a confidence score associated with the visual pattern, the dominant color, or a directionality present in the query image.

14. The device of claim 13, wherein the color histogram comprises a color distribution and relative spatial spans of the colors identified in the color distribution.

15. The device of claim 13, herein the orientation histogram comprises a measure of the directionality of spatial variation of the query image and a measure of complexity of the spatial variation.

16. The device of claim 13, further comprising receiving the query image from a mobile device, the query image comprising a photo taken by a user of an item, color, pattern, or texture of interest.

17. The device of claim 13, further comprising recommending a plurality of inventory items in response to receiving the query image, for display on a remote device, the plurality of inventory items comprising the inventory items most similar to the query image in accordance with the comparison of the first image attribute to the second image attribute for each of the respective plurality of inventory items.

18. A non-transitory computer readable medium including instructions, when executed by a processor, causes the processor to perform operations comprising:

determining a color histogram corresponding to a query image;

determining at least one of a visual pattern included in the query image, a dominant color of the query image, or an orientation histogram corresponding to the query image; and

comparing a first image attribute of the query image to a second image attribute of an inventory image corresponding to an inventory item, the first image attribute used in the comparison selected from among the color histogram, the dominant color, and the orientation histogram, and

wherein the selection of the first image attribute is based on a confidence score associated with the visual pattern, the dominant color, or a directionality present in the query image.

19. The non-transitory computer readable medium of claim 18, wherein the comparison of the first image attribute to the second image attribute comprises comparing the color histogram of the query image to a color histogram of the inventory image, if the confidence score of the determined visual pattern is above a pre-defined threshold, and wherein the comparison of the first image attribute to the second image attribute comprises comparing the color histogram of the query image to the color histogram of the inventory image, if no visual pattern is detected and the confidence score of the determined dominant color is below a pre-defined threshold.

20. The non-transitory computer readable medium of claim 18, wherein the comparison of the first image attribute to the second image attribute comprises comparing the dominant color of the query image to a dominant color of the inventory image, if no visual pattern is detected and the confidence score of the determined dominant color is above a pre-defined threshold, wherein the comparison of the first image attribute to the second image attribute comprises comparing the color histogram of the query image to a color histogram of the inventory image, if no visual pattern is detected and the confidence score of the directionality is below pre-defined threshold, and wherein the comparison of the first image attribute to the second image attribute comprises comparing the color histogram of the query image to the color histogram of the inventory image and comparing the orientation histogram of the query to an orientation histogram of the inventory image, if no visual pattern is detected and the confidence score of the directionality is above a pre-defined threshold.