Methods and apparatus for surface classification

• US 9,482,622 B2
• Filed: 04/02/2015
• Issued: 11/01/2016
• Est. Priority Date: 05/16/2014
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Sato, Munehiko, Raskar, Ramesh, Shi, Boxin, Olwal, Alex
• Primary Examiner(s): Chowdhury, Tarifur
• Assistant Examiner(s): Rahman, Md M
• Application Number:
  US14/677,898
• Publication Number:
  US 20150330905A1
• Time in Patent Office: 579 Days
• Field of Search: 356/73
• US Class Current: 1/1
• CPC Class Codes: G01N 2021/3155 Measuring in two spectral ranges, e.g. UV and visible
• CPC Class Codes: G01N 2021/3181 using LEDs
• CPC Class Codes: G01N 2021/479 Speckle
• CPC Class Codes: G01N 2021/8444 Fibrous material
• CPC Class Codes: G01N 21/251 Colorimeters; Construction thereof
• CPC Class Codes: G01N 21/255 Details, e.g. use of specially adapted sources, lighting or optical systems
• CPC Class Codes: G01N 21/474 Details of optical heads therefor, e.g. using optical fibres
• CPC Class Codes: G01N 21/84 Systems specially adapted for particular applications
• CPC Class Codes: G01N 2201/06113 Coherent sources; lasers
• CPC Class Codes: G01N 2201/0612 Laser diodes
• CPC Class Codes: G01N 2201/062 LED's
• CPC Class Codes: G01N 2201/0627 Use of several LED's for spectral resolution
• CPC Class Codes: G01N 2201/12 Circuits of general importance; Signal processing

Abstract

In illustrative implementations of this invention, light sources illuminate a surface with multi-spectral, multi-directional illumination that varies in direction, wavelength, coherence and collimation. One or more cameras capture images of the surface while the surface is illuminated under different lighting conditions. One or more computers take, as input, data indicative of or derived from the images, and determine a classification of the surface. Based on the computed classification, the computers output signals to control an I/O device, such that content displayed by the I/O device depends, at least in part, on the computed classification. In illustrative implementations, this invention accurately classifies a wide range of surfaces, including transparent surfaces, specular surfaces, and surfaces with few features.

Claims

1. A method comprising, in combination:

(a) a first set of one or more light sources illuminating an external surface of an object with coherent light during a first time period;

(b) a second set of one or more light sources illuminating the external surface with incoherent light during a second time period, such that the first time period and the second time period do not overlap;

(c) one or more cameras capturing images of the external surface

(i) during the first time period while the external surface is illuminated with the coherent light, and

(ii) during the second time period while the external surface is illuminated with the incoherent light; and

(d) one or more computers

(i) taking, as input, data indicative of or derived from the images,

(ii) determining a classification of the external surface, and

(iii) based on the classification, outputting signals to control an input/output (I/O) device, such that what content is displayed by the I/O depends, a least in part, on the classification;

wherein

(1) features are embedded in a transparent layer, which layer is illuminated by the coherent light during the first time period and by the incoherent light during the second time period,

(2) the features are invisible to an unaided human eye and have material properties that are different than the transparent layer, and

(3) the content that is displayed by the I/O device depends on data encoded by the features.

2. The method of claim 1, wherein direction and spectrum of illumination produced by the first set of light sources vary over time.

3. The method of claim 1, wherein:

(a) the one or more computers include a first computer and a second computer;

(b) at least the one or more cameras, the first and second sets of light sources, and the first computer are housed in a housing;

(c) the second computer is outside of the housing;

(d) the first computer takes, as an input, data from the one or more cameras, extracts parameter values from the data, and outputs the parameter values for wireless transmission to the second computer; and

(e) the second computer takes the parameter values as input, and computes a classification of the surface in real time.

4. The method of claim 1, wherein:

(a) the one or more cameras include a first camera and a second camera;

(b) the first and second cameras capture images of the surface; and

(c) the first camera has a spectral sensitivity that is different than the spectral sensitivity of the second camera.

5. The method of claim 3, wherein the parameters values, which are extracted by the first computer and are wirelessly transmitted to the second computer, include average pixel intensity of a frame.

6. The method of claim 1, wherein the surface that is classified is specular) The method of claim 3, wherein the parameter values, which are extracted from the data and are wirelessly transmitted to the second computer, include number of valid feature points in a frame.

7. An apparatus comprising, in combination:

(a) a first set of one or more coherent light sources for illuminating an external surface of an object with coherent light during a first time period;

(b) a second set of one or more incoherent light sources for illuminating the external surface with incoherent light during a second time period, such that the first time period and the second time period do not overlap;

(c) one or more cameras for capturing images of the external surface

(i) during the first time period while the external surface is illuminated with the coherent light, and

(ii) during the second time period while the external surface is illuminated with the incoherent light; and

(d) one or more computers programmed

(i) to take, as input, data indicative of or derived from the images, and

(ii) to determine a classification of the external surface, and based on the classification, to output signals to control an input/output (I/O) device, such that what content is displayed by the I/O device depends, at least in part, on the classification;

wherein

(1) features are embedded in a transparent layer, which layer is illuminated by the coherent light during the first time period and by the incoherent light during the second time period,

(2) the features are invisible to an unaided human eye and have material properties that are different than the transparent layer, and

(3) the content that is displayed by the I/O device depends on data encoded by the features.

8. The apparatus of claim 7, wherein the one or more computers are programmed to cause direction and spectrum of illumination produced by the first set of light sources to vary over time.

9. The apparatus of claim 7, wherein:

(a) the one or more computers include a first computer and a second computer;

(b) at least the one or more cameras, the first and second sets of light sources, and the first computer are housed in a housing;

(c) the second computer is outside of the housing;

(d) the first computer is programmed to extract parameter values from the images, and to output the parameter values for wireless transmission to the second computer; and

(e) the second computer is programmed to take the parameter values as input, and to compute a classification of the surface in real time.

10. The apparatus of claim 9, wherein the parameter values, which are extracted from the images by the first computer and are wirelessly transmitted to the second computer, include average pixel intensity of a frame.

11. The apparatus of claim 9, wherein the parameter values, which are extracted from the images by the first computer and are wirelessly transmitted to the second computer, include number of valid feature points in a frame.

12. The apparatus of claim 9, wherein, for a given frame, the parameter values, which are extracted from the images by the first computer and are wirelessly transmitted to the second computer:

(a) are represented by four bytes of wirelessly transmitted data; and

(b) include (i) average pixel intensity of the given frame, (ii) darkest pixel of the given frame, (iii) brightest pixel of the given frame, and (iv) number of valid feature points in the given frame.