Material recognition for object identification

• US 9,691,161 B1
• Filed: 09/25/2015
• Issued: 06/27/2017
• Est. Priority Date: 09/25/2015
• Status: Expired due to Fees

• Current Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Original Assignee: A9.com Incorporated (Amazon.com, Inc.)
• Inventors: Baldwin, Leo Benedict, Taylor, Colin Jon, Lewis, Isabella Talley, Yalniz, Ismet Zeki, Dhua, Arnab Sanat Kumar
• Primary Examiner(s): Lu, Tom Y
• Application Number:
  US14/866,677
• Time in Patent Office: 641 Days
• Field of Search: 382111, 382181
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/42 using transform domain methods
• CPC Class Codes: G06T 7/45 using co-occurrence matrix computation
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/141 Control of illumination
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/267 by performing operations on regions, e.g. growing, shrinking or watersheds
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

The recognition of objects such as clothing items can be improved by capturing image data that represents the material characteristics and true color of the object. A high resolution image can be captured that shows the individual threads and/or fibers, among other possible pattern or texture characteristics. Various approaches are discussed for determining the scale of these features to more accurately determine the way the material is made. Further, approaches are discussed that enable the true color of the material to be determined, through hardware and/or software, to further improve the accuracy of such a determination. The material and color properties can be combined with a conventional object recognition process to provide accurate object identification and/or authentication results.

Claims

1. A computer-implemented method, comprising:

capturing first image data including a first representation of an object, the object corresponding to an apparel item, the first representation providing a full view of the object from a first point of view;

receiving an instruction to capture second image data, the second image data including a second representation of a portion of the object enabling individual threads of the apparel item to be identified;

causing at least one instance of the second image data to be captured while light is emitted from at least one determined light source, each instance of the second image data captured with a respective set of illumination properties;

determining, from the second image data, weave pattern data for a material region of the apparel item;

determining, using the second image data, true color data, thread texture data, and dimension data for the threads of the material; and

providing the first image data, the weave pattern data, the true color data, the thread texture data, and the dimension data to an object recognition process, wherein the first image data is capable of being used to determine a type of the apparel item and the weave pattern data, true color data, and dimension data are capable of being used to discern between apparel items of the determined type.

2. The computer-implemented method of claim 1, further comprising:

establishing communications with an accessory having a determined size and a focal lens for enabling a camera to focus on the material region of the apparel item when the accessory is brought into contact with the apparel item, the accessory further shaped to prevent external light from being detected by the camera while positioned against the apparel item; and

causing one or more illumination sources to respectively emit light for each of at least a subset of instances of the second image data, each illumination source capable of emitting light with at least one of a determined wavelength, direction, or polarization.

3. The computer-implemented method of claim 1, further comprising:

determining a primary orientation of the threads of the material region of the apparel pattern; and

determining a set of features along the primary orientation as at least a portion of the weave pattern data.

4. The computer-implemented method of claim 1, further comprising:

analyzing the second image data to determine at least one type of thread, knot, fiber, or texton represented in the material region of the apparel item.

5. A computer-implemented method, comprising:

receiving a first image including a first representation of an item, the first representation of the item taken from at least a first determined distance enabling at least a majority of a first view of the item to be represented in the first image;

receiving a second image including a second representation of a portion of the item, the second representation taken from at most a second determined distance enabling a material texture of the item to be determined;

determining, using at least one processor of a computer system, a type of the item by analyzing the first image;

determining, using the at least one processor and data for the material texture from the second image, a matching item of the type of item that exhibits the material texture; and

providing information for the matching item as identifying information for the item represented in the first image.

6. The computer-implemented method of claim 5, further comprising:

determining, from the second image, a portion of the second image corresponding to a measurement standard, the measurement standard providing at least one of a true color reference or a distance reference.

7. The computer-implemented method of claim 6, further comprising:

analyzing the portion of the second image corresponding to the measurement standard to determine the represented color for each of a set of true color values; and

adjusting at least one color parameter of the second image based at least in part upon a difference between the represented colors in the second image and the true color values.

8. The computer-implemented method of claim 7, wherein the at least one color parameter includes at least one of a brightness, a gamma value, a white value, a contrast, or a color temperature.

9. The computer-implemented method of claim 6, further comprising:

analyzing the portion of the second image corresponding to the measurement standard to determine the represented length of the distance reference in the second image; and

adjusting at least one dimension of the second image based at least in part upon a difference between the represented length and the measurement standard.

10. The computer-implemented method of claim 5, further comprising:

determining a primary orientation of the material texture as represented in the second image; and

determining a set of features along the primary orientation as at least a portion of the material texture.

11. The computer-implemented method of claim 5, further comprising:

capturing at least a third image including a third representation of the portion of the item, the third image captured while using a different property of light than used to capture the second image; and

analyzing color data for the second image and the third image to determine true color data for the material texture.

12. The computer-implemented method of claim 11, wherein the different property includes at least one of a different light, orientation, polarization, wavelength, proximity, or color temperature.

13. The computer-implemented method of claim 5, further comprising:

receiving a plurality of material images, each material image associated with an item;

determining, for each of the material images, a true color histogram representing one or more true colors represented by at least one material region of the material image; and

storing data for each true color histogram with the respective material image, wherein an object identification process is enabled to utilize the true color histogram to match the corresponding item against data for a query image.

14. The computer-implemented method of claim 5, further comprising:

receiving tracking data indicating a change in at least one of a location, orientation, or state of a camera between capturing of the first image and the second image, the tracking data capable of being analyzed to determine relative scale information for the item represented in the first image and the material texture represented in the second image.

15. The computer-implemented method of claim 5, further comprising:

communicating with a device accessory for capturing the second image, wherein the device accessory is configured to emit light of a determined wavelength during the capturing of the second image while blocking out light from other sources.

16. A non-transitory computer readable storage medium storing instructions that, when executed by at least one processor of a computing device, cause the computing device to:

receive a first image including a first representation of an item, the first representation of the item taken from at least a first determined distance enabling at least a majority of a first view of the item to be represented in the first image;

receive a second image including a second representation of a portion of the item, the second representation taken from at most a second determined distance enabling a material texture of the item to be determined;

determine, using at least one processor of a computer system, a type of the item by analyzing the first image;

determine, using the at least one processor and data for the material texture from the second image, a matching item of the type of item that exhibits the material texture; and

provide information for the matching item as identifying information for the item represented in the first image.

17. The non-transitory computer readable storage medium of claim 16, wherein the instructions, when executed by the at least one processor, further cause the computing device to:

determine, from the second image, a portion of the second image corresponding to a measurement standard, the measurement standard providing at least one of a true color reference or a distance reference.

18. The non-transitory computer readable storage medium of claim 17, wherein the instructions, when executed by the at least one processor, further cause the computing device to:

analyze the portion of the second image corresponding to the measurement standard to determine the represented color for each of a set of true color values; and

adjust at least one color parameter of the second image based at least in part upon a difference between the represented colors in the second image and the true color values.

19. The non-transitory computer readable storage medium of claim 17, wherein the instructions, when executed by the at least one processor, further cause the computing device to:

analyze the portion of the second image corresponding to the measurement standard to determine the represented length of the distance reference in the second image; and

adjust at least one dimension of the second image based at least in part upon a difference between the represented length and the measurement standard.

20. The non-transitory computer readable storage medium of claim 16, wherein the instructions, when executed by the at least one processor, further cause the computing device to:

capture at least a third image including a third representation of the portion of the item, the third image captured while using a different property of light than used to capture the second image; and

analyze color data for the second image and the third image to determine true color data for the material texture.