Item recommendations using image feature data

• US 9,607,248 B2
• Filed: 10/22/2014
• Issued: 03/28/2017
• Est. Priority Date: 09/30/2011
• Status: Active Grant

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Bhardwaj, Anurag, Di, Wei, Hamid, Muhammad Raffay, Piramuthu, Robinson, Sundaresan, Neelakantan
• Primary Examiner(s): Dang, Duy M
• Application Number:
  US14/521,318
• Publication Number:
  US 20150055860A1
• Time in Patent Office: 888 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06Q 30/0629 for generating comparisons
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/41 based on statistical description of texture
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 10/56 relating to colour

Abstract

An apparatus and method to facilitate finding recommendations for a query image are disclosed herein. A color histogram is determined corresponding to the query image. Determining at least one of a visual pattern included in the query image, a dominant color of the query image, or an orientation histogram corresponding to the query image. Performing comparison of a first image attribute of the query image to a second image attribute of an inventory image corresponding to an inventory item, wherein the first image attribute used in the comparison is selected from among the color histogram, the dominant color, and the orientation histogram. The selection of the first image attribute is based on a confidence score associated with the visual pattern, the dominant color, or a directionality present in the query image.

Claims

1. A method comprising:

determining, by one or more processors of a machine, a color histogram of a query image;

determining, by one or more processors of the machine, a first image attribute of the query image;

calculating, by one or more processors of the machine, a confidence score of the first image attribute, the confidence score quantifying a degree to which the first image attribute is present in the query image; and

comparing, by one or more processors of the machine, the first image attribute of the query image to a second image attribute of an item image that depicts an item, the comparing of the first and second image attributes being based on the confidence score of the first image attribute of the query image.

2. The method of claim 1, wherein:

the first image attribute is a first visual pattern depicted in the query image;

the second image attribute is a second visual pattern depicted in the item image; and

the comparing of the first and second image attributes compares the first and second visual patterns based on the confidence score of the first visual pattern.

3. The method of claim 1, wherein:

the first image attribute is a first dominant color depicted in the query image;

the second image attribute is a second dominant color depicted in the item image; and

the comparing of the first and second image attributes compares the first and second dominant colors based on the confidence score of the first dominant color.

4. The method of claim 1, wherein:

the first image attribute is a first orientation histogram of the query image;

the second image attribute is a second orientation histogram of the item image; and

the comparing of the first and second image attributes compares the first and second orientation histograms based on the confidence score of the first orientation histogram.

5. The method of claim 1, wherein:

the comparing of the first and second image attributes is based on the confidence score of the first image attribute transgressing a predefined threshold value that corresponds to the first image attribute.

6. The method of claim 1, wherein:

the first image attribute is the color histogram of the query image; and

the second image attribute is a color histogram of the item image.

7. The method of claim 6, wherein:

the comparing of the first and second image attributes compares the color histogram of the query image with the color histogram of the item image based on a failure to detect a visual pattern in the query image.

8. The method of claim 6, further comprising:

calculating an orientation confidence score based on an orientation histogram of the query image; and

wherein

the comparing of the first and second image attributes compares the color histogram of the query image with the color histogram of the item image based on the orientation confidence score failing to transgress a predefined threshold value.

9. The method of claim 1, wherein:

the determining of the color histogram of the query image is based on information contained only within a sampling mask area applied to the query image.

10. The method of claim 9, wherein:

the sampling mask area defines a central portion of the item image and defines a central portion of the query image.

11. The method of claim 1, further comprising:

receiving the query image from a device that generated the query image by photographing a subject that exhibits the first image attribute.

12. The method of claim 1, further comprising:

recommending the item depicted in the item image, in response to reception of the query image and based on the comparing of the first and second image attributes.

13. A system comprising:

a color histogram module comprising one or more processors of a machine and configured to determine a color histogram of a query image;

an image attribute module comprising one or more processors of the machine and configured to;

determine a first image attribute of the query image; and

calculate a confidence score of the first image attribute, the confidence score quantifying a degree to which the first image attribute is present in the query image; and

a comparison module comprising one or more processors of the machine and configured to compare the first image attribute of the query image to a second image attribute of an item image that depicts an item, the comparing of the first and second image attributes being based on the confidence score of the first image attribute of the query image.

14. The system of claim 13, wherein:

the first image attribute is at least one of a first visual pattern depicted in the query image, a first dominant color of the query image, or a first orientation histogram of the query image; and

the second image attribute is at least one of a second visual pattern depicted in the item image, a second dominant color of the item image, or a second orientation histogram of the item image.

15. The system of claim 13, wherein:

the comparison module is configured to compare of the first and second image attributes based on the confidence score of the first image attribute transgressing a predefined threshold value that corresponds to the first image attribute.

16. The system of claim 13, further comprising:

an image capture module comprising one or more processors of the machine and configured to receive the query image from a device that generated the query image by photographing a subject that exhibits the first image attribute.

17. The system of claim 13, wherein:

the comparison module comprising one or more processors of the machine is configured to recommend the item depicted in the item image, in response to reception of the query image and based on the comparing of the first and second image attributes.

18. A non-transitory machine-readable storage medium comprising instructions that, when executed by processors of a machine, cause the machine to perform operations comprising:

determining, by one or more processors of the machine, a color histogram of a query image;

determining, by one or more processors of the machine, a first image attribute of the query image; and

comparing, by one or more processors of the machine, the first image attribute of the query image to a second image attribute of an item image that depicts an item, the comparing of the first and second image attributes being based on a confidence score of the first image attribute of the query image, the confidence score quantifying a degree to which the first image attribute is present in the query image.

19. The non-transitory machine-readable storage medium of claim 18, wherein the operations further comprise:

calculating, by one or more processors of the machine, the confidence score of the first image attribute.

20. The non-transitory machine readable medium of claim 18, wherein:

the first image attribute is at least one of a first visual pattern depicted in the query image, a first dominant color of the query image, or a first orientation histogram of the query image; and

the second image attribute is at least one of a second visual pattern depicted in the item image, a second dominant color of the item image, or a second orientation histogram of the item image.