MATERIAL RECOGNITION FROM AN IMAGE

• US 20110243450A1
• Filed: 04/01/2010
• Published: 10/06/2011
• Est. Priority Date: 04/01/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Liu, Ce
• Granted Patent: US 8,565,536 B2
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06F 18/2111 by using evolutionary computational techniques, e.g. genetic algorithms
• CPC Class Codes: G06F 18/295 Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/85 Markov-related models; Markov random fields
• CPC Class Codes: G06V 20/80 Recognising image objects characterised by unique random patterns

Abstract

A method of operating a computer system to perform material recognition based on multiple features extracted from an image is described. A combination of low-level features extracted directly from the image and multiple novel mid-level features extracted from transformed versions of the image are selected and used to assign a material category to a single image. The novel mid-level features include non-reflectance based features such as the micro-texture features micro jet and micro-SIFT and the shape feature curvature, and reflectance-based features including edge slice and edge ribbon. An augmented Latent Dirichlet Allocation (LDA) model is provided as an exemplary Bayesian framework for selecting a subset of features useful for material recognition of objects in an image.

Claims

1. A method of categorizing a material of an object in an image, the method comprising:

extracting, with at least one processor, a plurality of features from the image;

combining at least two of the plurality of features to generate a model comprising distributions for the at least two of the plurality of features across groups of pixels in the image; and

categorizing the material of the object in the image based, at least in part, on the distributions in the model.

2. The method of claim 1, wherein categorizing comprises using a Bayesian framework.

3. The method of claim 1, wherein categorizing comprises using at least one pattern recognition algorithm selected from a group consisting of a support vector machine algorithm, a nearest neighbor algorithm, and a boosting algorithm.

4. The method of claim 1, wherein the least two of the plurality of features comprises at least one reflectance-based feature and at least one non-reflectance based feature.

5. The method of claim 4, wherein the at least one non-reflectance based feature comprises at least one texture-based feature.

6. The method of claim 5, wherein the at least one texture-based feature comprises a scale-invariant feature transform (SIFT).

7. The method of claim 1, wherein the at least one reflectance-based feature comprises an edge slice feature representing image changes along a direction normal to at least one edge in the image.

8. The method of claim 7, wherein the image changes comprise changes in a distribution of gradient orientations determined for a group of pixels oriented normal to the at least one edge.

9. The method of claim 1, wherein combining at least two of the plurality of features comprises:

selecting the at least two of the plurality of features to combine based, at least in part, on a trained model in which combinations of features are analyzed to determine the at least two of the plurality of features.

10. The method of claim 1, wherein the at least two of the plurality of features used to generate the model are less than all of the plurality of features extracted from the image.

11. A computer-readable storage medium encoded with a plurality of instructions that, when executed by a computer, perform a method of:

extracting at least one reflectance-based feature and at least one non-reflectance based feature from an image; and

categorizing a material in the image based, at least in part, on the at least one reflectance-based feature and the at least one non-reflectance based feature.

12. The computer-readable storage medium of claim 11, wherein the at least one reflectance-based feature comprises an edge slice feature representing image changes along a direction normal to at least one edge in the image.

13. The computer-readable storage medium of claim 11, wherein the at least one reflectance-based feature comprises a edge ribbon feature that describes image changes along a direction tangent to at least one edge in the image.

14. The computer-readable storage medium of claim 11, wherein that at least one non-reflectance based feature is a texture-based feature.

15. The computer-readable storage medium of claim 11, wherein the method further comprises:

combining a first dictionary created for the at least one reflectance-based feature and a second dictionary created for the at least one non-reflectance based feature to generate a joint dictionary;

training a bag-of-words model using the joint dictionary; and

wherein the material in the image is categorized based, at least in part, on the trained bag-of-words model.

16. A computer system, comprising:

at least one processor programmed to;

extract a plurality of features from a plurality of images assigned to a plurality of material categories, wherein the plurality of features comprises at least one reflectance-based feature and at least one non-reflectance based feature; and

select a subset of the plurality of features, wherein the subset of the plurality of features is determined by a learning procedure in which combinations of features are analyzed to determine the subset of the plurality of features.

17. The computer system of claim 16, wherein the at least one processor is further programmed to:

quantize each of the features in the selected subset of the plurality of features;

combine the quantized features in a bag-of-words model; and

assign a material category to the image based, at least in part, on the bag-of-words model.

18. The computer system of claim 16, wherein the learning procedure is a greedy algorithm that considers contributions of each of the plurality of features in a stepwise manner.

19. The computer system of claim 16, wherein the selected subset of features comprises an edge slice feature representing image changes along a direction tangent to at least one edge in the image.

20. The computer system of claim 20, wherein the image changes comprise changes in a distribution of gradient orientations determined for a group of pixels oriented normal to the at least one edge.