SENSOR-SYNCHRONIZED SPECTRALLY-STRUCTURED-LIGHT IMAGING

• US 20140293091A1
• Filed: 03/08/2014
• Published: 10/02/2014
• Est. Priority Date: 05/21/2012
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rhoads, Geoffrey B., Brunk, Hugh L., Lord, John D.
• Granted Patent: US 9,593,982 B2
• Time in Patent Office: Days
• US Class Current: 348/234
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/6898 Portable consumer electronic devices, e.g. music players, telephones, tablet computers
• CPC Class Codes: G01J 3/0229 using masks, aperture plates, spatial light modulators or spatial filters, e.g. reflective filters
• CPC Class Codes: G01J 3/0272 Handheld
• CPC Class Codes: G01J 3/10 Arrangements of light sources specially adapted for spectrometry or colorimetry
• CPC Class Codes: G01J 3/2823 Imaging spectrometer
• CPC Class Codes: G01J 3/36 Investigating two or more bands of a spectrum by separate detectors
• CPC Class Codes: G01J 3/447 Polarisation spectrometry
• CPC Class Codes: G01J 3/51 using colour filters
• CPC Class Codes: G01J 3/513 having fixed filter-detector pairs
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2431 Multiple classes
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 9/69 for modifying the colour signals by gamma correction controlling camera response for colour signals H04N23/83
• CPC Class Codes: H04N 9/73 Colour balance circuits, e.g. white balance circuits or colour temperature control camera processing pipelines for colour balance H04N23/88

Abstract

A smartphone is adapted for use as an imaging spectrometer, by synchronized pulsing of different LED light sources as different image frames are captured by the phone'"'"'s CMOS image sensor. A particular implementation employs the CIE color matching functions, and/or their orthogonally transformed functions, to enable direct chromaticity capture. A great variety of other features and arrangements are also detailed.

Claims

1. A method of spectral imaging comprising:

obtaining color images through a sensor, comprising R, G and B color components, the color images comprising pixel values corresponding to locations in a field of view of the sensor, and being captured during corresponding illumination periods in which different LED light sources illuminate the field of view of the sensor, and the color images represent a combination of light from an LED light source and ambient light reflected from a scene in the field of view;

applying a coupling factor to pixel values of the color images corresponding to a spectral channel to provide spectral channel values, wherein the spectral channel is associated with a combination of light source and a color component;

summing pixel values for corresponding locations to provide sums for the corresponding locations; and

for N spectral channels and the locations, computing a ratio of the spectral channel value and sum of a corresponding location to provide a spectricity vector for the corresponding location.

2. The method of claim 1 comprising obtaining a reference image of ambient light without illumination by the LED light sources, and subtracting pixel values of the reference image from pixel values of a color image obtained under illumination of an LED light source.

3. The method of claim 1 further including reversing gamma from the color images prior to applying the coupling factor.

5. The method of claim 1 wherein the sensor comprises a plenoptic sensor, and further comprising obtaining depth information per pixel, or a plurality of sub-images of a scene at different viewing angles.

6. The method of claim 1 further including computing a spatial relationship function of pixels sampled from different locations of a scene to obtain a vector of multi-spectral channels per pixel and spatial relationships among the pixels, and inputting the vector to a classifier.

7. The method of claim 6 wherein the scene comprises at least one produce item, and applying the classifier to the vector to classify the produce item.

8. The method of claim 6 wherein the spatial relationship function comprises a function of multi-spectral values of pixels at 2 or more spatial dimensions.

9. The method of claim 8 wherein the spatial relationship function comprises a function of multi-spectral values of pixels at 3 or more spatial dimensions.

10. The method of claim 7, further including sampling polarimetric information from the produce item, and computing a function of polarimetric information and spectral information to produce a polarimetric dependent vector, and inputting the polarimetric dependent vector to a classifier, and classifying the produce item based on the polarimetric dependent vector.

11. The method of claim 10 further including sampling the polarimetric information at and below a surface of a produce item.

12. The method of claim 1 further including obtaining pixels sampled from different locations of a scene and combining multi-channel spectral values of each pixel to obtain a vector, and classifying the vector to ascertain a type of object from which the pixels are sampled.

13. The method of claim 1 further including obtaining pixels sampled from different locations of an object, each pixel comprising an N-D spectral vector, and classifying each of a plurality of N-D spectral vectors separately to ascertain a type of object from which each pixel is sampled, and combining output of the classifying for pixels from the object to ascertain the type of object.

14. The method of claim 1 further including obtaining pixels sampled from different locations of an object, each pixel comprising an N-D spectral vector, combining the pixels into a combined N-D spectral vector, and classifying each of a plurality of combined N-D spectral vectors separately to ascertain a type of object from which each combined N-D spectral vector is obtained, and combining output of the classifying for combined N-D spectral vectors from the object to ascertain the type of object.

15. The method of claim 14 wherein the combining output comprises assessing similarity between a probability distribution of classifier output for combined N-D spectral vectors and probability distribution of classifier output of a training set of known objects.

16. The method of claim 7 wherein the spatial relationship function comprises differences between N-D spectricity pixels sampled from a produce item.

4. (canceled)

17. A method comprising:

obtaining pixels sampled from a scene, the pixels each comprising an N-D spectral vector; and

classifying the N-D spectral vector to ascertain class of an object in the scene.

18. The method of claim 17 wherein the obtaining comprises sampling pixels under different illumination to obtain N different spectral samples per pixel.

19. The method of claim 17 wherein obtaining comprises obtaining the pixels from a plenoptic sensor, and further comprising obtaining depth information per pixel, or a plurality of sub-images of a scene at different viewing angles.

20. The method of claim 17 further including computing a spatial relationship function of pixels sampled from different locations of the object to obtain a vector of multi-spectral channels per pixel and spatial relationships among the pixels, inputting the vector to a classifier, and classifying the object based on the vector.

21-30. -30. (canceled)

31. An apparatus comprising:

an imaging apparatus for obtaining pixels sampled from a scene, the pixels each comprising an N-D spectral vector;

the imaging apparatus configured to sample pixels under different illumination to obtain N different spectral samples per pixel; and

a classifier in communication with the imaging apparatus, for classifying the N-D spectral vector to ascertain class of an object in the scene.

32-43. -43. (canceled)