Clothing search in images

• US 8,798,362 B2
• Filed: 08/15/2011
• Issued: 08/05/2014
• Est. Priority Date: 08/15/2011
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Wang, Xianwang, Zhang, Tong
• Primary Examiner(s): Alavi, Amir
• Application Number:
  US13/209,901
• Publication Number:
  US 20130044944A1
• Time in Patent Office: 1,086 Days
• Field of Search: 382/103, 382/108, 382115-118, 382/162, 382/164, 382/165, 382/173, 382/181, 382/195, 382/199, 382218-220, 348/222.1, 348/223.1, 348/E05.031, 345/158
• US Class Current: 382/165
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 18/24323 Tree-organised classifiers
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

Methods, systems, and computer readable media with executable instructions, and/or logic are provided for clothing search in images. An example method of clothing search in images can include characterizing clothing within a plurality of reference images using a processor, and characterizing clothing within a query image using a processor. A number of the plurality of reference images having clothing with similar color features as clothing of the query image is identified using a processor. A subset of the identified number of the plurality of reference images having clothing with predefined non-color attributes as clothing of the query image are selected using a processor.

Claims

1. A method for clothing search in images, comprising:

characterizing, using a processor, clothing within a plurality of reference images;

characterizing, using a processor, clothing within a query image;

identifying, using a processor, a first subset of the plurality of reference images having clothing with similar color features as clothing of the query image; and

selecting, using a processor, a second subset derived from the identified first subset of the plurality of reference images, the second subset including those plurality of reference images having clothing with similar predefined clothing attributes other than color as clothing of the query image.

2. The method of claim 1, further comprising constructing, using a processor, a customized color codebook based on the plurality of reference images, wherein characterizing clothing includes

detecting a body feature in an image;

extracting a clothing region relative to the body feature;

removing skin-colored regions from the clothing region;

classifying a number of predefined clothing attributes other than color of the clothing region; and

determining a dominant color feature associated with the clothing region.

3. The method of claim 2, wherein constructing a customized color codebook includes:

obtaining a plurality of color samples from across the clothing regions extracted from the plurality of reference images; and

identifying dominant colors for the extracted clothing regions based on the plurality of color samples.

4. The method of claim 3, wherein constructing a customized color codebook for the clothing region includes:

extracting a number of dominant color 7×

7 pixel non-overlapping patches from segmented clothing; and

performing illumination normalization on each dominant color patch.

5. The method of claim 2, wherein detecting the body feature includes detecting a face.

6. The method of claim 2, wherein detecting the body feature includes detecting a limb.

7. The method of claim 2, wherein removing skin-colored regions from the clothing region includes applying a dynamic threshold to determine skin color, the dynamic threshold being determined from the body feature.

8. The method of claim 2, wherein extracting the clothing region relative to the body feature includes extracting a bounding box adjacent to and located immediately below the body feature.

9. The method of claim 8, wherein detecting the body feature includes detecting a face, and the bounding box width is based on a width of the face, and the bounding box height is based on the height of the face.

10. The method of claim 8, wherein extracting the clothing region includes segmenting clothing according to a Grabcut method, the bounding box being used to initialize the Grabcut method.

11. The method of claim 2, further comprising:

training, using a processor, binary classifiers with the predefined clothing attributes other than color of the query image; and

computing, using a processor, one or more similar color features for each clothing of the plurality of reference images and query image with a customized color codebook,

wherein selecting the second subset of the identified first subset of the plurality of reference images having clothing with similar predefined clothing attributes other than color as clothing of the query image includes selecting clothing with the trained binary classifiers and matching clothing in the query image with the identified clothing from the plurality of reference images using the computed one or more similar color features.

12. The method of claim 1, wherein identifying the first subset of the plurality of reference images having clothing with similar color features as clothing of the query image includes measuring color similarity using a Jeffrey Divergence.

13. A non-transitory computer-readable medium having computer-executable instructions stored thereon, the computer-executable instructions comprising instructions that, if executed by one or more processors, cause the one or more processors to:

receive a query image;

construct a customized color codebook based on a plurality of reference images;

characterize clothing within the plurality of reference images for predefined clothing attributes other than color and one or more color features associated with the constructed customized color codebook;

characterize clothing within the query image for predefined clothing attributes other than color and one or more color features associated with the constructed customized color codebook;

identify a number of the plurality of reference images having clothing with similar color features as clothing of the query image; and

select a subset of the identified number of the plurality of reference images having clothing with similar predefined clothing attributes other than color as clothing of the query image.

14. The non-transitory computer-readable medium of claim 13, further comprising instructions that, if executed by one or more processors, cause the one or more processors to:

for each of the plurality of reference images and the query image;

detect a body feature in an image;

extract a clothing region relative to the body feature;

remove skin-colored regions from the clothing region;

classify a number of predefined clothing attributes other than color of the clothing region; and

determine one or more color features associated with the clothing region.

15. A computing system, comprising:

a non-transitory computer-readable medium having computer-executable instructions stored thereon; and

a processor coupled to the non-transitory computer-readable medium, wherein the processor executes the instructions to;

construct a customized color codebook based on a plurality of reference images;

characterize clothing within the plurality of reference images;

characterize clothing within a query image;

identify a number of the plurality of reference images having clothing with similar color features as clothing of the query image; and

select a subset of the identified number of the plurality of reference images having clothing with similar predefined clothing attributes other than color as clothing of the query image,

wherein the processor executes the instructions to characterize clothing includes instructions to;

detect a body feature in an image;

extract a clothing region relative to the body feature;

remove skin-colored regions from the clothing region;

classify a number of predefined clothing attributes other than color of the extracted clothing region; and

determine color features of the extracted clothing region.