Utilizing color descriptors to determine color content of images

• US 9,841,877 B2
• Filed: 02/05/2016
• Issued: 12/12/2017
• Est. Priority Date: 05/13/2013
• Status: Active Grant

• Current Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Original Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Inventors: Arora, Himanshu, Dhua, Arnab Sanat Kumar
• Primary Examiner(s): BEKELE, MEKONEN T
• Application Number:
  US15/017,156
• Publication Number:
  US 20160155025A1
• Time in Patent Office: 676 Days
• Field of Search: 382162, 382170, 382305, 382165, 382173, 382171, 707 3, 707100, 707E17108, 705 27
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/56 relating to colour

Abstract

Various embodiments provide a method for determining color information for an image. For example, a color descriptor for an image can be determined and compared against color descriptors stored for each of a number of sample images, which each represent a color in a color space. Upon comparison, matching scores can be generated for a color match between the image and each respective sample image. In this example, the number of sample images with a matching score above a threshold value can be summed and the image can be assigned to a color associated with a highest frequency of the number of sample images. Accordingly, the assigned color of the image can then be used in a “query by color” search or a browse-by-color capability.

Claims

1. A computer-implemented method, comprising:

receiving a query for an item offered through an electronic marketplace, the query including an image that includes a representation of the item and an indication of a product type associated with the item;

analyzing the image to identify an area of the image that includes the representation of the item, the area of the image including a plurality of pixels;

determining a closest fine color for individual pixels of the plurality of pixels, individual fine colors being associated with a color family;

computing a fine color histogram and a family color histogram for the area based at least in part on the closest fine color for individual pixels and associated color family;

computing a color descriptor for the representation of the item based at least in part on a combination of the fine color histogram and the family color histogram;

comparing the color descriptor to an electronic catalog of color descriptors to determine a color class of the representation of the item;

providing the color class and product type to an identification service, the identification service used to generate a ranking score based and least in part on a matching of the color class to a respective color class associated with a respective item of a subset of the items offered by the electronic marketplace; and

retrieving the subset of the items for display, each item of the subset of the items being arranged based at least in part on a respective color class.

2. The computer-implemented method of claim 1, further comprising:

displaying a plurality of color filters with the subset of the items;

receiving a selection of one of the plurality of color filters to enable a color filter corresponding to a color class; and

displaying items of the subset of the items associated with a color class matching the color class of the color filter.

3. The computer-implemented method of claim 1, further comprising:

determining supplemental content associated with a color class matching the color class of the item offered through the electronic marketplace; and

displaying, with the subset of the items, the supplemental content.

4. The computer-implemented method of claim 1, wherein the image that includes the representation of the item is received from a mobile device interface.

5. The computer-implemented method of claim 1, further comprising:

identifying a number of highest scoring sample images associated with a respective color matching score above a threshold matching score;

grouping each of the sample images of the number of highest scoring sample images into a class color cluster;

summing color matching scores for each of the sample images in each class color cluster; and

associating the image to the color class associated with a class color cluster having a color matching score sum above a threshold value.

6. The computer-implemented method of claim 5, further comprising:

selecting a plurality of sample images to represent each color class of a color space;

determining respective color descriptors for each of the plurality of sample images; and

storing the color descriptors for the plurality of sample images in the electronic catalog of color descriptors to compare to the image for the color descriptor for the image.

7. The computer-implemented method of claim 6, wherein at least one sample image represents a color near a boundary between two adjacent color classes.

8. The computer-implemented method of claim 6, wherein grouping each of the sample images further includes:

comparing a color descriptor of each sample image of the plurality of sample images to color descriptors of the number of highest scoring sample images to determine a similarity between each of the sample images.

9. A computing device, comprising:

a processor; and

memory including instructions that, when executed by the processor, cause the computing device to;

receive a query for an item offered through an electronic marketplace, the query including an image that includes a representation of the item and an indication of a product type associated with the item;

analyze the image to identify an area of the image that includes the representation of the item, the area of the image including a plurality of pixels;

determine a closest fine color for individual pixels of the plurality of pixels, individual fine colors being associated with a color family;

compute a fine color histogram and a family color histogram for the area based at least in part on the closest fine color for individual pixels and associated color family;

compute a color descriptor for the representation of the item based at least in part on a combination of the fine color histogram and the family color histogram;

compare the color descriptor to an electronic catalog of color descriptors to determine a color class of the representation of the item;

provide the color class and product type to an identification service, the identification service used to generate a ranking score based and least in part on a matching of the color class to a respective color class associated with a respective item of a subset of the items offered by the electronic marketplace; and

retrieve the subset of the items for display, each item of the subset of the items being arranged based at least in part on a respective color class.

10. The computing device of claim 9, wherein the instructions, when executed by the processor, further enable the computing device to:

identify a number of highest scoring sample images associated with a respective color matching score above a threshold matching score;

group each of the sample images of the number of highest scoring sample images into a class color cluster;

sum the color matching scores for each of the sample images in each class color cluster to determine a plurality of color matching scores; and

associate the image to the color class associated with a class color cluster having a color matching score sum above a threshold value.

11. The computing device of claim 10, wherein the instructions, when executed by the processor, further enable the computing device to:

compute a dot product between the color descriptor of the image and each of the color descriptors for the number of highest scoring sample images to generate the plurality of color matching scores.

12. The computing device of claim 9, wherein the instructions, when executed by the processor, further enable the computing device to:

display a plurality of color filters with the subset of the items;

receive a selection of one of the plurality of color filters;

enable a color filter based at least in part on the selection; and

display items of the subset of the items associated with a color class matching a color class of the color filter.

13. The computing device of claim 9, wherein the instructions, when executed by the processor, further enable the computing device to:

determine supplemental content associated with a color class matching the color class of the item offered through the electronic marketplace; and

display, with the subset of the items, the supplemental content.

14. The computing device of claim 9, wherein the image that includes the representation of the item is received from a mobile device interface.

15. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, enable a computing device to:

receive a query for an item offered through an electronic marketplace, the query including an image that includes a representation of the item and an indication of a product type associated with the item;

analyze the image to identify an area of the image that includes the representation of the item, the area of the image including a plurality of pixels;

determine a closest fine color for individual pixels of the plurality of pixels, individual fine colors being associated with a color family;

compute a fine color histogram and a family color histogram for the area based at least in part on the closest fine color for individual pixels and associated color family;

compute a color descriptor for the representation of the item based at least in part on a combination of the fine color histogram and the family color histogram;

compare the color descriptor to an electronic catalog of color descriptors to determine a color class of the representation of the item;

provide the color class and product type to an identification service, the identification service used to generate a ranking score based and least in part on a matching of the color class to a respective color class associated with a respective item of a subset of the items offered by the electronic marketplace; and

retrieve the subset of the items for display, each item of the subset of the items being arranged based at least in part on a respective color class.

16. The non-transitory computer-readable storage medium of claim 15, wherein the instructions that, when executed by the at least one processor, further enable the computing device to:

display a plurality of color filters with the subset of the items;

receive a selection of one of the plurality of color filters;

enable a color filter based at least in part on the selection; and

display items of the subset of the items associated with a color class matching a color class of the color filter.

17. The non-transitory computer-readable storage medium of claim 15, wherein the instructions that, when executed by the at least one processor, further enable the computing device to:

determine supplemental content associated with a color class matching the color class of the item offered through the electronic marketplace; and

display, with the subset of the items, the supplemental content.

18. The non-transitory computer-readable storage medium of claim 15, wherein the instructions that, when executed by the at least one processor, further enable the computing device to:

identify a number of highest scoring sample images associated with a respective color matching score above a threshold matching score;

group each of the sample images of the number of highest scoring sample images into a class color cluster;

sum the color matching scores for each of the sample images in each class color cluster; and

associate the image to the color class associated with a class color cluster having a color matching score sum above a threshold value.