MATERIAL CLASSIFICATION USING OBJECT/MATERIAL INTERDEPENDENCE WITH FEEDBACK

• US 20150144537A1
• Filed: 11/26/2013
• Published: 05/28/2015
• Est. Priority Date: 11/26/2013
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: SKAFF, Sandra
• Granted Patent: US 9,082,071 B2
• Time in Patent Office: Days
• US Class Current: 209/577
• CPC Class Codes: B07C 5/34 Sorting according to other particular properties material testing per se G01N; quality control G07C3/14
• CPC Class Codes: B07C 5/342 according to optical properties, e.g. colour according to radiation transmittivity B07C5/3416
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06V 10/464 using a plurality of salient features, e.g. bag-of-words [BoW] representations
• CPC Class Codes: G06V 10/58 relating to hyperspectral data

Abstract

The present disclosure relates to classification of a material type of an object. A first phase applies an object classifier and a material classifier to obtain first object and material probabilities of the object. A second phase applies an interdependent object/material classifier to the first object and material probabilities to obtain further object and material class probabilities. The interdependent object/material classifier performs multiple iterations of calculating the further object and material class probabilities, and utilizes feedback in which an immediately preceding calculated prior further object class probability is included in a next iteration of calculating a further material class probability, and an immediately preceding calculated prior further material class probability is included in a next iteration of calculating a further object class probability.

Claims

1. A method of classifying an object, comprising:

a first phase comprising application of an object classifier to the object to obtain first object class probabilities of the object belonging to respective different object classes, and application of a material classifier to the object to obtain first material class probabilities of a material type of the object belonging to respective different material classes; and

a second phase comprising application of an interdependent object/material classifier to the first object probabilities to obtain further object class probabilities, and to the first material class probabilities to obtain further material class probabilities, so as to classify the object class and to classify the material type of the object,

wherein, the interdependent object/material classifier performs multiple iterations of calculating the further object class probabilities and multiple iterations of calculating the further material class probabilities, and

wherein, an immediately preceding calculated prior further object class probability is included in a next iteration of calculating a further material class probability, and an immediately preceding calculated prior further material class probability is included in a next iteration of calculating a further object class probability.

2. The method according to claim 1, further comprising, following each iteration of calculating the further object class probability, and each iteration of calculating the further material class probability, determining whether the respective object class and material class probabilities meet a predetermined probability value.

3. The method according to claim 2, wherein the iterations of the object/material classifier are terminated when it is determined that material class meets the predetermined probability value.

4. The method according to claim 2, wherein, in a case where it is determined that the predetermined probability value is not met, the interdependent object/material classifier continues the object class iterations and the material class iterations sequentially until a) it is determined that the value has been met, b) a predetermined number of iterations has been met, or c) a predetermined change amount in the probability between iterations has been met.

5. The method according to claim 1, wherein the first object class probabilities and the first material class probabilities are based on training data.

6. The method according to claim 1, wherein further probabilities calculated by the interdependent object/material classifier are calculated using a Bayesian approach.

7. The method according to claim 1, wherein the first object class probabilities and the first material class probabilities are calculated using a technique selected from a group which includes support vector machines (SVM), neural networks, Gaussian mixture modeling, and K-means clustering.

8. The method according to claim 1, wherein the object classifier in the first phase comprises derivation of descriptors followed by object classification using the descriptors.

9. The method according to claim 8, wherein the derivation of descriptors comprises derivation of low level descriptors including color and scale-invariant-feature-transformation (SIFT).

10. The method according to claim 8, wherein the derivation of descriptors includes derivation of high level descriptors including one of bag-of-visual-words (BOV) descriptors and Fisher vectors (FV).

11. The method according to claim 1, wherein the object classifier in the first phase is selected from a group which includes discriminative classification and generative classification.

12. The method according to claim 1, wherein the material classifier in the first phase comprises discriminative classification in which bidirectional reflectance distribution function (BRDF) features are extracted from the object followed by support vector machine (SVM) material classification based on the extracted BRDF features.

13. The method according to claim 1, wherein the material classifier in the first phase comprises discriminative classification in which spectral features are extracted from the object based on estimations of spectral reflectance from the object followed by probability calculations as to most likely material classification based on the extracted spectral features.

14. The method according to claim 1, wherein, the interdependent object/material classifier comprises application of a pre-defined set of object/material interdependent weighting functions for calculating the further object class probabilities, and application of a pre-defined set of material/object interdependent functions for calculating the further material class probabilities, and

wherein the pre-defined set of object/material interdependent weighting functions define a likelihood of a particular object being more or less likely to belong to a particular material class.

15. The method according to claim 1, wherein, the interdependent object/material classifier comprises application of a pre-defined set of object/material interdependent weighting functions for calculating the further object class probabilities, and application of a pre-defined set of material/object interdependent functions for calculating the further material class probabilities, and

wherein the pre-defined set of material/object interdependent weighting functions define a likelihood of a particular material being more or less likely to belong to a particular object class.

16. A method of recycling objects, comprising the steps of:

conveying recyclable objects to a material classification position;

capturing an image of the recyclable objects;

classifying, according to the method of claim 1, the object class and the material class of the object in the captured image; and

sorting the recyclable objects according to their respective object and material classes as classified in the classifying step.

17. A system for recycling of objects, comprising:

a conveyor system for conveying recyclable objects to a material classification position;

an image capture device for capturing an image of the recyclable objects;

a classifying device for classifying, using the method according to claim 1, the object class and the material class of the object in the captured image; and

a sorting device for sorting the recyclable objects according to their respective object and material classes classified by the classifying device.

18. A non-transitory computer readable storage medium in which is stored computer executable code of a program that, when executed by a processor of a computer, causes the computer to execute the method according to claim 1.