Material sorting using a vision system

• US 10,710,119 B2
• Filed: 04/26/2018
• Issued: 07/14/2020
• Est. Priority Date: 07/18/2016
• Status: Active Grant

• Current Assignee: Sortera Technologies Incorporated
• Original Assignee: UHV Technologies Incorporated
• Inventors: Kumar, Nalin, Garcia, Jr., Manuel Gerardo, Tyagi, Kanishka
• Primary Examiner(s): Fox, Charles A
• Assistant Examiner(s): Kumar, Kalyanavenkateshware
• Application Number:
  US15/963,755
• Publication Number:
  US 20180243800A1
• Time in Patent Office: 810 Days
• CPC Class Codes: B07C 2501/0054 Sorting of waste or refuse
• CPC Class Codes: B07C 5/3416 according to radiation transmissivity, e.g. for light, x-rays, particle radiation detecting hidden objects G01V
• CPC Class Codes: B07C 5/3422 using video scanning devices, e.g. TV-cameras
• CPC Class Codes: B07C 5/368 actuated independently
• CPC Class Codes: G01N 2223/643 object on conveyor
• CPC Class Codes: G01N 23/223 by irradiating the sample with X-rays or gamma-rays and by measuring X-ray fluorescence
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

A material sorting system sorts materials utilizing a vision system that implements a machine learning system in order to identify or classify each of the materials, which are then sorted into separate groups based on such an identification or classification. The material sorting system may include an x-ray fluorescence system to perform a classification of the materials in combination with the vision system, whereby the classification efforts of the vision system and x-ray fluorescence system are combined in order to classify and sort the materials.

Claims

1. A system for classifying and sorting a first heterogeneous mix of materials comprising:

a first device configured to produce image data of the first heterogeneous mix of materials;

a first conveyor system configured to convey the first heterogeneous mix of materials past the first device;

a first data processing system comprising a machine learning system configured to assign a first classification to a first one of the materials based on the image data of the first heterogeneous mix of materials, wherein the first classification is based on a first knowledge base containing a previously generated library of observed characteristics captured from a homogenous set of samples of the first one of the materials; and

a first sorter configured to sort the first one of the materials from the first heterogeneous mix of materials as a function of the first classification of the first one of the materials.

2. The system as recited in claim 1, wherein the library of observed characteristics were captured by a camera configured to capture images of the homogenous set of samples of the first one of the materials as they were conveyed past the camera.

3. The system as recited in claim 1, wherein the first device is a camera configured to capture visual images of the materials to produce the image data, and wherein the observed characteristics are visually observed characteristics.

4. The system as recited in claim 3, further comprising:

an x-ray source configured to illuminate the materials;

an x-ray fluorescence detector configured to detect x-ray fluorescence spectra from the materials; and

circuitry configured to assign a second classification to the first one of the materials as a function of the detected x-ray fluorescence spectra, wherein the sorting by the first sorter of the first one of the materials from the first heterogeneous mix of materials is performed as a function of a combination of the first and second classifications.

5. The system as recited in claim 1, further comprising:

an x-ray source configured to illuminate the materials;

an x-ray fluorescence detector configured to detect x-ray fluorescence spectra from the materials; and

circuitry configured to convert the detected x-ray fluorescence spectra into the image data.

6. The system as recited in claim 1, wherein the sorting by the first sorter of the first one of the materials from the first heterogeneous mix of materials produces a second heterogeneous mix of materials that comprises the first heterogeneous mix of materials minus the sorted first one of the materials, the system further comprising:

a second device configured to produce image data of the second heterogeneous mix of materials;

a second conveyor system configured to convey the second heterogeneous mix of materials past the second device;

a second data processing system comprising a machine learning system configured to assign a second classification to a second one of the materials based on the image data of the second heterogeneous mix of materials, wherein the second classification is based on a second knowledge base containing a previously generated library of observed characteristics captured from a homogenous set of samples of the second one of the materials; and

a second sorter configured to sort the second one of the materials from the second heterogeneous mix of materials as a function of the second classification of the second one of the materials.

7. The system as recited in claim 1, wherein the sorting by the first sorter of the first one of the materials from the first heterogeneous mix of materials results in a plurality of pieces of the first one of the materials, the system further comprising:

a second device configured to produce image data of the plurality of pieces of the first one of the materials;

a second conveyor system configured to convey the plurality of pieces of the first one of the materials past the second device after the plurality of pieces of the first one of the materials has been sorted by the first sorter from the first heterogeneous mix of materials;

a second data processing system comprising a machine learning system configured to assign a second classification to certain ones of the plurality of pieces of the first one of the materials based on the image data of the plurality of pieces of the first one of the materials, wherein the second classification is based on a second knowledge base containing a previously generated library of observed characteristics captured from a homogenous set of samples of the certain ones of the plurality of pieces of the first one of the materials; and

a second sorter configured to sort the certain ones of the plurality of pieces of the first one of the materials from the plurality of pieces of the first one of the materials as a function of the second classification.

8. The system as recited in claim 7, wherein the plurality of pieces of the first one of the materials includes one or more pieces of wrought aluminum and one or more pieces of cast aluminum, wherein the second classification distinguishes wrought aluminum from cast aluminum so that the second sorter is configured to sort the one or more pieces of wrought aluminum from the one or more pieces of cast aluminum.

9. The system as recited in claim 7, wherein the sorting by the first sorter of the first one of the materials from the first heterogeneous mix of materials produces a second heterogeneous mix of materials that comprises the first heterogeneous mix of materials minus the sorted first one of the materials, the system further comprising:

a third device configured to produce image data of the second heterogeneous mix of materials;

a third conveyor system configured to convey the second heterogeneous mix of materials past the third device after the first one of the materials has been sorted by the first sorter from the first heterogeneous mix of materials;

a third data processing system comprising a machine learning system configured to assign a third classification to a second one of the materials within the second heterogeneous mix of materials based on the image data of the second heterogeneous mix of materials, wherein the third classification is based on a third knowledge base containing a previously generated library of observed characteristics captured from a homogenous set of samples of the second one of the materials; and

a second sorter configured to sort the second one of the materials from the second heterogeneous mix of materials as a function of the second classification of the second one of the materials within the second heterogeneous mix of materials.

10. The system as recited in claim 1, further comprising a chemical sensor configured to determine at least one chemical element within one or more pieces of the first heterogeneous mix of materials, wherein the first sorter is configured to sort the first one of the materials from the first heterogeneous mix of materials as a function of a combination of the first classification of the first one of the materials and the at least one chemical element determined by the chemical sensor.

17. The system as recited in claim 1, wherein the first classification is assigned to the first one of the materials without a benefit of an analysis based on irradiating the first heterogeneous mix of materials with an x-ray source.

18. The system as recited in claim 1, wherein the first heterogeneous mix of materials includes one or more pieces of wrought aluminum and one or more pieces of cast aluminum, wherein the first classification distinguishes wrought aluminum from cast aluminum so that the first sorter is configured to sort the one or more pieces of wrought aluminum from the one or more pieces of cast aluminum.

19. The system as recited in claim 1, further comprising:

a camera configured to produce image data of the homogenous set of samples of the first one of the materials; and

the machine learning system configured to produce the first knowledge base containing the previously generated library of observed characteristics from the image data captured from the homogenous set of samples of the first one of the materials.

20. The system as recited in claim 1, wherein the machine learning system implements one or more machine learning algorithms configured to perform the assigning of the first classification to the first one of the materials as a function of the first knowledge base, wherein the first knowledge base contains parameters configured during a training stage to visually recognize the observed characteristics, wherein the training stage is configured to process a control sample of a plurality of the homogenous set of samples of the first one of the materials through the machine learning system in order to create the knowledge base.

11. A device for identifying at least one characteristic of a material, comprising:

an x-ray source configured to illuminate the material to produce an x-ray fluorescence spectrum from the material;

an x-ray fluorescence detector configured for recoding the x-ray fluorescence spectrum from the material into x-ray fluorescence data that is characteristic of the material;

an optical sensor configured to capture visual image data of the material; and

a processing unit configured with a machine learning system configured to identify a characteristic of the material from the x-ray fluorescence data and/or the visual image data wherein the machine learning system is configured with a neural network trained to compare the captured visual image data of the material with a library of visually observed characteristics captured from images of a homogenous set of material samples all possessing the at least one characteristic.

12. The device as recited in claim 11, wherein the machine learning system comprises one or more algorithms configured to identify the characteristic of the material from the x-ray fluorescence data and/or the visual image data, wherein the one or more algorithms are selected from the group consisting of a support vector machine, nearest neighbor, naï

ve Bayes, decision trees, linear regression, clustering, k- means clustering, association rules, q-learning, temporal difference, deep adversarial network, gradient descent, polynomial regression, learning curves, regularized learning model, logistic regression, svm regression, nonlinear svm, cart algorithm, random forests, boosting, stacking, projection, manifold learning, principal component analysis, dimensionality reduction, cart training, autoencoders, reinforcement learning, and any combination thereof.

13. A method for extracting a characteristic of a first object within a moving stream of objects, comprising:

detecting a location of the first object relative to the moving stream of objects;

illuminating the first object with x-rays;

recording an x-ray fluorescence spectrum emanating from the first object;

capturing a visual image of the first object; and

utilizing a machine learning system to identify the characteristic of the first object based on either the x-ray fluorescence spectrum or the visual image of the first object, or a combination thereof, wherein the machine learning system is configured with a neural network trained to compare the captured visual image of the first object with a library of visually observed characteristics captured from visual images of a homogenous set of objects all possessing the characteristic.

14. The method as recited in claim 13, wherein the characteristic of the first object is folds in the first object.

15. The method as recited in claim 14, further comprising redirecting the first object from the stream of objects as a function of the identification of the characteristic of the first object.

16. A method for extracting a characteristic of a first object within a moving stream of objects, comprising:

detecting a location of the first object relative to the moving stream of objects;

illuminating the first object with x-rays;

recording an x-ray fluorescence spectrum emanating from the first object;

capturing a visual image of the first object;

utilizing a machine learning system to identify the characteristic of the first object based on either the x-ray fluorescence spectrum or the visual image of the first object, or a combination thereof, wherein the characteristic of the first object is folds in the first object, wherein the machine learning system comprises an artificial intelligence neural network; and

redirecting the first object from the stream of objects as a function of the identification of the characteristic of the first object.