Material recognition from an image

• US 9,025,866 B2
• Filed: 06/27/2013
• Issued: 05/05/2015
• Est. Priority Date: 04/01/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Liu, Ce
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US13/929,651
• Publication Number:
  US 20130294685A1
• Time in Patent Office: 677 Days
• Field of Search: 382/159, 382224-228
• US Class Current: 382/159
• CPC Class Codes: G06F 18/2111 by using evolutionary computational techniques, e.g. genetic algorithms
• CPC Class Codes: G06F 18/295 Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/85 Markov-related models; Markov random fields
• CPC Class Codes: G06V 20/80 Recognising image objects characterised by unique random patterns

Abstract

A method of operating a computer system to perform material recognition based on multiple features extracted from an image is described. A combination of low-level features extracted directly from the image and multiple novel mid-level features extracted from transformed versions of the image are selected and used to assign a material category to a single image. The novel mid-level features include non-reflectance based features such as the micro-texture features micro-jet and micro-SIFT and the shape feature curvature, and reflectance-based features including edge slice and edge ribbon. An augmented Latent Dirichlet Allocation (LDA) model is provided as an exemplary Bayesian framework for selecting a subset of features useful for material recognition of objects in an image.

Claims

1. A computer system comprising:

a processing device, and

a memory device storing computer-executable instructions which, when executed by the processing device, cause the processing device to;

extract a plurality of features from a plurality of images assigned to a plurality of material categories, wherein the plurality of features comprises at least one reflectance-based feature and at least one non-reflectance based feature; and

select a subset of the plurality of features, wherein the subset of the plurality of features is selected by a learning procedure in which combinations of features are analyzed to determine the subset of the plurality of features,

wherein the learning procedure is a greedy algorithm that considers contributions of each of the plurality of features in a stepwise manner.

2. The computer system of claim 1, wherein the greedy algorithm is configured to select additional features for the subset of the plurality of features one at a time.

3. The computer system of claim 2, wherein the greedy algorithm is further configured to determine separate material recognition rates for the additional features.

4. The computer system of claim 3, wherein the greedy algorithm is further configured to select the additional features until an individual material recognition rate of the separate material recognition rates is less than a previous individual material recognition rate.

5. The computer system of claim 4, wherein the greedy algorithm is further configured to select the subset of the plurality of features based on maximizing the material recognition rate.

6. A method implemented by one or more computer processing devices, the method comprising:

receiving a training set of images assigned to multiple material categories;

extracting multiple features from the training set of images; and

analyzing combinations of the multiple features using a learning procedure to select a subset of the multiple features,

wherein the learning procedure is a greedy algorithm that considers contributions of individual features of the multiple features one at a time.

7. The method of claim 6, further comprising:

dividing the training set of images into a first portion used by the learning procedure for training a model and a second portion for evaluation of the model.

8. The method of claim 7, further comprising evaluating the model by determining a material recognition rate of the model for the second portion of the training set of images.

9. The method of claim 8, further comprising receiving an initial material recognition rate and comparing the determined material recognition rate to the initial material recognition rate to select the subset of the multiple features.

10. The method of claim 6, the method further comprising:

repeating at least the analyzing for different material categories.

11. A hardware computer-readable storage medium storing computer readable instructions that, when executed by one or more processing devices, cause the one or more processing devices to perform acts comprising:

receiving a set of images assigned to multiple material categories, the images of the set having multiple image features;

dividing the set of images into a training set and an evaluation set;

joining individual dictionaries of the multiple image features to create a combined dictionary;

using the combined dictionary to train a recognition model on the training set to obtain a trained recognition model; and

determining a material recognition rate for the trained recognition model based on recognition of individual material categories in the evaluation set by the trained recognition model.

12. The hardware computer-readable storage medium of claim 11, the training the model further comprising:

extracting the multiple image features from the training set;

quantizing individual extracted features;

combining the quantized individual extracted features into a single document model;

analyzing combinations of the quantized individual extracted features using a learning procedure to select a subset of the multiple image features, wherein the trained recognition model is obtained using the subset of the multiple image features; and

assigning an individual material category to another image by comparing the single document model to one or more pre-stored feature distributions.