Visual search utilizing color descriptors

• US 9,704,033 B2
• Filed: 06/23/2015
• Issued: 07/11/2017
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Original Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Inventors: Ramesh, Sunil, Arora, Himanshu, Dhua, Arnab Sanat Kumar
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): YOUSSEF, MENATOLLAH M
• Application Number:
  US14/747,331
• Publication Number:
  US 20150286863A1
• Time in Patent Office: 749 Days
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06V 10/56 relating to colour

Abstract

Various embodiments provide a method for computing color descriptors of product images. For example, a number of fine color representatives can be determined to describe color variation in an image as a histogram by assigning a saturation value and a brightness value to a plurality of color hues. For each pixel of the image, the closest color among a defined fine color representative set is computed. In this example, each of the pixels is assigned a color ID corresponding to their closest matching fine color representative and at least one family color ID corresponding one or more pure color families. In this example, a histogram of the color representatives and a histogram for the color families are computed. A single color vector descriptor for the image is then determined by combining the family histogram with the color representative histogram.

Claims

1. A computer-implemented method, comprising:

under the control of one or more computer systems configured with executable instructions,

defining a number of color families, each color family including a plurality of color values;

determining a color value of the plurality of color values for representative colors associated with pixels of an image, the color value comprising a first color within a threshold distance of a second color that corresponds to the representative colors;

assigning the representative colors to one or more color families of the number of color families, wherein the assigning is based at least in part on the threshold distance; and

combining histograms of the representative colors and the one or more color families to provide a descriptor of color variation for the image, wherein the descriptor is at least a basis for digital searching of the image.

2. The computer-implemented method of claim 1, wherein each color family includes a plurality of red, green, blue (RGB) color values, each color value is a RGB color value, and the color space is an RGB color space.

3. The computer-implemented method of claim 1, wherein at least one representative color in the representative colors is assigned to two color families.

4. The computer-implemented method of claim 1, further comprising:

using the number of color families to determine color similarity between a first image and the image.

5. The computer-implemented method of claim 4, wherein the step of using the number of color families to determine color similarity further comprises determining a similarity measure between a first vector color descriptor and a second vector color descriptor.

6. The computer-implemented method of claim 5, wherein the similarity measure is determined by computing a dot product of the first vector color descriptor and the second vector color descriptor.

7. The computer-implemented method of claim 6, wherein each of the first vector color descriptor and the second vector color descriptor is computed based on a global histogram of the color representatives and a family histogram of the color families.

8. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause a computing device to:

define a number of color families, each color family including a plurality of color values;

determine a color value of the plurality of color values for each of a set of representative colors associated with pixels of an image, the color value comprising a first color within a threshold distance of a second color that corresponds to the representative colors;

assign the representative colors to one or more color families of the number of color families, wherein the assigning is based at least in part on the threshold distance; and

combine histograms of the representative colors and the one or more color families to provide a descriptor of color variation for the image, wherein the descriptor is at least a basis for digital searching of the image.

9. The non-transitory computer-readable storage medium of claim 8, wherein each color family includes a plurality of red, green, blue (RGB) color values, each color value is an RGB color value, and the color space is an RGB color space.

10. The non-transitory computer-readable storage medium of claim 8, wherein at least one representative color in the representative colors is assigned to two color families.

11. The non-transitory computer-readable storage medium of claim 8, wherein the instructions that, when executed by the at least one processor, further cause the computing device to:

determine, based on the number of color families, color similarity between a first image and the image.

12. The non-transitory computer-readable storage medium of claim 11, wherein the color similarity is determined by determining a similarity measure between a first vector color descriptor and a second vector color descriptor.

13. The non-transitory computer-readable storage medium of claim 12, wherein the similarity measure is determined by computing a dot product of the first vector color descriptor and the second vector color descriptor.

14. The non-transitory computer-readable storage medium of claim 13 wherein each of the first vector color descriptor and the second vector color descriptor is computed based on a global histogram of the color representatives and a family histogram of the color families.

15. A computing device, comprising:

a processor;

a display screen; and

memory including instructions that, when executed by the processor, cause the computing device to;

define a number of color families, each color family including a plurality of color values;

determine a color value of the plurality of color values for each of a set of representative colors associated with pixels of an image, the color value comprising a first color within a threshold distance of a second color that corresponds to the representative colors;

assign the representative colors to one or more color families of the number of color families, wherein the assigning is based at least in part on the threshold distance; and

combine histograms of the representative colors and the one or more color families to provide a descriptor of color variation for the image, wherein the descriptor is at least a basis for digital searching of the image.

16. The computing device of claim 15, wherein each color family includes a plurality of red, green, blue (RGB) color values, each color value is an RGB color value, and the color space is an RGB color space.

17. The computing device of claim 15, wherein at least one representative color in the representative colors is assigned to two color families.

18. The computing device of claim 15, wherein the instructions, when executed by at least one processor, further cause the computing device to:

use the number of color families to determine color similarity between a first image and the image.

19. The computing device of claim 18, wherein the color similarity is determined by determining a similarity measure between a first vector color descriptor and a second vector color descriptor.

20. The computing device of claim 19, wherein the similarity measure is determined by computing a dot product of the first vector color descriptor and the second vector color descriptor.