Utilizing color descriptors to determine color content of images

• US 9,299,009 B1
• Filed: 05/13/2013
• Issued: 03/29/2016
• Est. Priority Date: 05/13/2013
• Status: Active Grant

• Current Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Original Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Inventors: Arora, Himanshu, Dhua, Arnab Sanat Kumar
• Primary Examiner(s): BEKELE, MEKONEN T
• Application Number:
  US13/892,666
• Time in Patent Office: 1,051 Days
• Field of Search: 382/162, 382/170, 382/305, 382/165, 382/173, 382/171, 707/3, 707/100, 707/E17.108, 705/27
• US Class Current: 1/1
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/56 relating to colour

Abstract

Various embodiments provide a method for determining color information for an image. For example, a color descriptor for an image can be determined and compared against color descriptors stored for each of a number of sample images, which each represent a color in a color space. Upon comparison, matching scores can be generated for a color match between the image and each respective sample image. In this example, the number of sample images with a matching score above a threshold value can be summed and the image can be assigned to a color associated with a highest frequency of the number of sample images. Accordingly, the assigned color of the image can then be used in a “query by color” search or a browse-by-color capability.

Claims

1. A computer-implemented method, comprising:

defining a color space comprising a plurality of color classes;

selecting a plurality of sample images for each color class of the color space;

associating each one of the sample images with a color descriptor of a plurality of color descriptors;

obtaining an image;

determining a closest fine color for each of at least a portion of pixels of the image, each fine color being associated with a color family;

computing a fine color histogram and a family color histogram for the image;

determining a color descriptor of the plurality of color descriptors based at least in part on a combination of the fine color histogram and the family color histogram for the image;

comparing the color descriptor to each color descriptor associated with each one of the plurality of sample images for each color class to generate a plurality of color matching scores;

identifying a determined number of highest scoring sample images having a color matching score above a threshold matching score;

grouping each of the sample images of the determined number of highest scoring sample images into a class color cluster;

summing the color matching scores for each of the sample images in each class color cluster; and

assigning the image to at least one color class for at least one class color cluster having a color matching score sum above a threshold value.

2. The computer-implemented method of claim 1, further comprising:

selecting the plurality of sample images to represent each color class of the color space;

determining the color descriptors for each of the plurality of sample images; and

storing the color descriptors for the plurality of sample images in a database to compare to the image for determining color information for the image.

3. The computer-implemented method of claim 1, further comprising:

choosing at least one sample image to represent a color near a boundary between two adjacent color classes.

4. The computer-implemented method of claim 1, wherein clustering each of the sample images includes:

comparing a color descriptor of each sample image to the color descriptors of each other sample image of the determined number of highest scoring sample images to determine a similarity between each of the sample images.

5. The computer-implemented method of claim 4, wherein the image is assigned to two or more color classes based at least on in part on the matching score sum for each of the two or more color classes being above the threshold value.

6. A computer-implemented method, comprising:

defining a color space comprising a plurality of color classes;

selecting a plurality of representative images for each color class of the color space;

associating each one of the representative images with a color descriptor of a plurality of color descriptors;

obtaining an image;

determining a closest fine color for each of at least a portion of pixels of the image, each fine color being associated with a color family;

computing a fine color histogram and a family color histogram for the image;

determining a color descriptor for the image from a combination of the fine color histogram and the family color histogram for the image;

comparing the color descriptor against each color descriptor associated with each one of the plurality of representative images for each color class to generate a plurality of matching scores;

identifying representative images, from the plurality of representative images, having a matching score above a threshold value;

summing the matching scores by color class; and

assigning the image to a color class having a highest matching score sum.

7. The computer-implemented method of claim 6, further comprising:

computing a dot product between the color descriptor of the image and each of the color descriptors for the plurality of representative images to generate the plurality of matching scores.

8. The computer-implemented method of claim 6, wherein the image is a product image for a product available for purchase from an electronic marketplace.

9. The computer-implemented method of claim 6, further comprising:

clustering each of the identified representative images by color class; and

comparing a color descriptor of each identified representative image to the color descriptors of each other identified representative image to determine a similarity between each of the identified representative images.

10. The computer-implemented method of claim 9, wherein the image is assigned to two or more color classes based at least on in part on the matching score sum for each of the two or more color classes being above the threshold value.

11. The computer-implemented method of claim 6, wherein the plurality of representative images each represent a fine color in a color space divided into a plurality of color families, each color family being represented by at least one fine color.

12. The computer-implemented method of claim 6, further comprising:

choosing a representative image to represent a color near a boundary between two adjacent color classes.

13. The computer-implemented method of claim 11, further comprising:

selecting the plurality of representative images to represent each color class;

determining the color descriptors for the plurality of representative images; and

storing the color descriptors for the plurality of representative images in a database to compare to the image for determining color information for the image.

14. A computing device, comprising:

a processor; and

memory including instructions that, when executed by the processor, cause the computing device to;

define a color space comprising a plurality of color classes;

select a plurality of representative images for each color class of the color space;

associate each one of the representative images with a color descriptor of a plurality of color description;

obtain an image;

determine a closest fine color for each of at least a portion of pixels of the image, each fine color being associated with a color family;

compute a fine color histogram and a family color histogram for the image;

determine a color descriptor for the image from a combination of the fine color histogram and the family color histogram for the image;

compare the color descriptor against each color descriptor associated with each one of the plurality of representative images for each color class to generate a plurality of matching scores;

identify representative images, from the plurality of representative images, having a matching score above a threshold value;

sum the matching scores by color class; and

assign the image to a color class having a highest matching score sum.

15. The computing device of claim 14, wherein the instructions, when executed by the processor, further enable the computing device to:

compute a dot product between the color descriptor of the image and each of the color descriptors for the plurality of representative images to generate the plurality of matching scores.

16. The computing device of claim 14, wherein the image is a product image for a product available for purchase from an electronic marketplace.

17. The computing device of claim 14, wherein the instructions, when executed by the processor, further enable the computing device to:

cluster each of the identified representative images by color class; and

compare a color descriptor of each identified representative image to the color descriptors of each other identified representative images to determine a similarity between each of the identified representative images.

18. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause a computing device to:

define a color space comprising a plurality of color classes;

select a plurality of representative images for each color class of the color space;

associate each one of the representative images with a color descriptor of a plurality of color description;

obtain an image;

determine a closest fine color for each of at least a portion of pixels of the image, each fine color being associated with a color family;

compute a fine color histogram and a family color histogram for the image;

determine a color descriptor for the image from a combination of the fine color histogram and the family color histogram for the image;

compare the color descriptor against each color descriptor associated with each one of the plurality of representative images for each color class to generate a plurality of matching scores;

identify representative images, from the plurality of representative images, having a matching score above a threshold value;

sum the matching scores by color class; and

assign the image to a color class having a highest matching score sum.

19. The non-transitory computer-readable storage medium of claim 18, wherein the instructions that, when executed by the at least one processor, further cause the computing device to:

cluster each of the identified representative images by color class; and

compare a color descriptor of each identified representative image to the color descriptors of each other identified representative images to determine a similarity between each of the identified representative images.

20. The non-transitory computer-readable storage medium of claim 19, wherein the image is assigned to two or more color classes based at least on in part on the matching score sum for each of the two or more color classes being above the threshold value.

21. The non-transitory computer-readable storage medium of claim 18, wherein the plurality of representative images each represent a fine color in a color space divided into a plurality of color families, each color family being represented by at least one fine color.

22. The non-transitory computer-readable storage medium of claim 21, wherein the instructions that, when executed by the at least one processor, further cause the computing device to:

choose a representative image to represent a first fine color near a boundary between two adjacent color families in the color space.

23. The non-transitory computer-readable storage medium of claim 21, wherein the instructions that, when executed by the at least one processor, further cause the computing device to:

select the plurality of representative images to represent each color family of the color space;

determine the color descriptors for the plurality of representative images; and

store the color descriptors for the plurality of representative images in a database to compare to the image for determining color information for the image.