Extraction of image feature data from images

• US 9,235,859 B2
• Filed: 05/23/2014
• Issued: 01/12/2016
• Est. Priority Date: 09/30/2011
• Status: Active Grant

• Current Assignee: eBay Inc.
• Original Assignee: eBay Inc.
• Inventors: Bhardwaj, Anurag, Di, Wei, Hamid, Muhammad Raffay, Piramuthu, Robinson, Sundaresan, Neelakantan
• Primary Examiner(s): Dang, Duy M
• Application Number:
  US14/286,594
• Publication Number:
  US 20140254927A1
• Time in Patent Office: 599 Days
• Field of Search: 382/165
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06Q 30/0629 for generating comparisons
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/41 based on statistical description of texture
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 10/56 relating to colour

Abstract

An apparatus and method for obtaining image feature data of an image are disclosed herein. A color histogram of the image is extracted from the image, the extraction of the color histogram including performing one-dimensional sampling of pixels comprising the image in each of a first dimension of a color space, a second dimension of the color space, and a third dimension of the color space. An edge map corresponding to the image is analyzed to detect a pattern included in the image. In response to a confidence level of the pattern detection being below a pre-defined threshold, extracting from the image an orientation histogram of the image. And identify a dominant color of the image.

Claims

1. A method comprising:

generating a color histogram of an image by performing one-dimensional samplings of pixels of the image in each dimension of a multi-dimensional color space;

performing pattern detection on an edge map of the image;

generating an orientation histogram of the image in response to a confidence level of the pattern detection failing to transgress a threshold value, the generating of the orientation histogram being performed by a processor of a machine; and

determining a dominant color of the image.

2. The method of claim 1, wherein:

the multi-dimensional color space includes a hue dimension, a saturation dimension, and a value dimension.

3. The method of claim 1, wherein:

the multi-dimensional color space includes a first dimension, a second dimension, and a third dimension; and

the performing of the one-dimensional samplings includes sampling a first pixel among the pixels of the image in the first dimension in 24 bins, sampling the first pixel in the second dimension in 8 bins, and sampling the first pixel in the third dimension in 8 bins.

4. The method of claim 3, wherein:

a second pixel among the pixels of the image has a saturation that fails to transgress a predetermined threshold; and

the performing of the one-dimensional samplings includes sampling the second pixel in a fourth dimension distinct from the first, second, and third dimensions of the multi-dimensional color space.

5. The method of claim 1, wherein:

the multi-dimensional color space includes a first dimension, a second dimension, and a third dimension; and

the generating of the color histogram includes applying a first weight to a one-dimensional sampling of a first pixel among the pixels of the image in the first dimension, a second weight to a one-dimensional sampling of the first pixel in the second dimension, and a third weight to a one-dimensional sampling of the first pixel in the third dimension.

6. The method of claim 5, wherein:

a second pixel among the pixels of the image has a saturation that fails to transgress a predetermined threshold; and

the generating of the color histogram includes applying a fourth weight to a one-dimensional sampling of the second pixel in a fourth dimension distinct from the first, second, and third dimensions of the multi-dimensional color space.

7. The method of claim 5, wherein:

the generating of the color histogram includes combining a plurality of weighted one-dimensional samples of the pixels of the image to form the color histogram.

8. The method of claim 1, wherein:

the determining of the dominant color of the image determines that the dominant color is present on more spatial area within a sample area of the image than other colors within the sample area of the image.

9. The method of claim 1, wherein:

the determining of the dominant color of the image includes clustering colors of a sample area of the image and identifying a largest color cluster among a plurality of colors of the sample area of the image.

10. The method of claim 9, wherein:

the multi-dimensional color space is a first multi-dimensional color space; and

the determining of the dominant color of the image includes clustering the colors of the sample area in accordance with a second multi-dimensional color space different from the first multi-dimensional color space.

11. The method of claim 10, wherein:

the second multi-dimensional space is a nonlinear color space adopted by the International Commission on Illumination.

12. The method of claim 1, wherein:

the performing of the one-dimensional samplings of the pixels includes performing uniform one-dimensional samplings of the pixels.

13. The method of claim 1, wherein:

the performing of the pattern detection includes calculating a circularity value of a blob within the edge map of the image.

14. The method of claim 1 further comprising:

determining that the orientation histogram is indicative of the image having low spatial variation; and

wherein

the generating of the color histogram is in response to the orientation histogram indicating that the image has low spatial variation.

15. The method of claim 1 further comprising:

determining that the orientation histogram is indicative of the image having high spatial variation;

determining a confidence score of an orientation of the image; and

indexing the image in accordance with the confidence score of the orientation of the image.

16. The method of claim 1, wherein:

the image depicts a sellable item.

17. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

generating a color histogram of an image by performing one-dimensional samplings of pixels of the image in each dimension of a multi-dimensional color space;

performing pattern detection on an edge map of the image;

generating an orientation histogram of the image in response to a confidence level of the pattern detection failing to transgress a threshold value; and

determining a dominant color of the image.

18. The non-transitory machine-readable storage medium of claim 17, wherein:

the generating of the orientation histogram includes;

calculating an x-derivative and a y-derivative of the edge map;

calculating a gradient and an orientation based on the x-derivative and the y-derivative; and

applying a weight to each edge pixel in the edge map to obtain the orientation histogram.

19. A system comprising:

one or more processors;

a color histogram module that configures at least one processor among the one or more processors to generate a color histogram of an image by performing one-dimensional samplings of pixels of the image in each dimension of a multi-dimensional color space;

a pattern module that configures at least one processor among the one or more processors to perform pattern detection on an edge map of the image;

an orientation histogram module that configures at least one processor among the one or more processors to generate an orientation histogram of the image in response to a confidence level of the pattern detection failing to transgress a threshold value; and

a dominant color module that configures at least one processor among the one or more processors to determine a dominant color of the image.

20. The system of claim 19 further comprising:

a capture module configured to receive the image from a device communicatively coupled to the system.