Sensor-synchronized spectrally-structured-light imaging

• US 10,498,941 B2
• Filed: 07/16/2018
• Issued: 12/03/2019
• Est. Priority Date: 05/21/2012
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rhoads, Geoffrey B., Brunk, Hugh L., Lord, John D.
• Primary Examiner(s): Nazrul, Shahbaz
• Application Number:
  US16/036,651
• Publication Number:
  US 20190014245A1
• Time in Patent Office: 505 Days
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/6898 Portable consumer electronic devices, e.g. music players, telephones, tablet computers
• CPC Class Codes: G01J 3/0229 using masks, aperture plates, spatial light modulators or spatial filters, e.g. reflective filters
• CPC Class Codes: G01J 3/0272 Handheld
• CPC Class Codes: G01J 3/10 Arrangements of light sources specially adapted for spectrometry or colorimetry
• CPC Class Codes: G01J 3/2823 Imaging spectrometer
• CPC Class Codes: G01J 3/36 Investigating two or more bands of a spectrum by separate detectors
• CPC Class Codes: G01J 3/447 Polarisation spectrometry
• CPC Class Codes: G01J 3/51 using colour filters
• CPC Class Codes: G01J 3/513 having fixed filter-detector pairs
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2431 Multiple classes
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 9/69 for modifying the colour signals by gamma correction controlling camera response for colour signals H04N23/83
• CPC Class Codes: H04N 9/73 Colour balance circuits, e.g. white balance circuits or colour temperature control camera processing pipelines for colour balance H04N23/88

Abstract

A spectral imaging device is configured to capture color images synchronized with controlled illumination from different color light emitting diodes. A processor in the device applies a coupling factor to sampled color images to convert sampled pixels into spectral channels corresponding to LED color and color filter. Multi-spectral spectricity vectors produced at pixel locations are used along with spatial information to classify objects, such as produce items.

Claims

1. An apparatus for spectral imaging comprising:

an image sensor operable to obtain images, the images comprising pixel values corresponding to locations in a field of view of the image sensor;

an illumination source comprising at least three LEDs, each a different color;

a drive circuit that selectively strobes on the LEDs synchronized with capture of frames of the images by the image sensor;

means for obtaining multi-spectral channel values per pixel corresponding to the LEDs;

means for computing a spatial relationship function of pixels sampled from different locations of the field of view; and

means for classifying an object depicted in the images as being one of plural different classes based on the multi-spectral channels per pixel and spatial relationship function.

2. The apparatus of claim 1 wherein the object comprises a produce item, and the means for classifying identifies a type of produce.

3. The apparatus of claim 1 wherein the spatial relationship function comprises a function of multi-spectral values of pixels at 2 or more spatial dimensions.

4. The apparatus of claim 3 wherein the spatial relationship function comprises a function of multi-spectral values of pixels at 3 or more spatial dimensions.

5. The apparatus of claim 1 wherein the image sensor comprises a polarizer, and the apparatus comprises a processor configured with instructions to detect pixels in the images that are impacted by specular reflection based on polarization obtained from the polarizer.

6. The apparatus of claim 1 wherein the image sensor comprises a monochrome sensor.

7. The apparatus of claim 6 wherein the illumination source comprises at least four different LEDs, each a different color, and each of the frames is captured to isolate a different state of LED illumination corresponding to the selective strobing of a color of one of the LEDs.

8. The apparatus of claim 1 wherein the means for classifying comprises a processor configured with instructions to execute a vector quantizer on the multi-spectral channel values per pixel.

9. The apparatus of claim 8 wherein the means for classifying comprises a processor configured with instructions to execute a tree structured vector classifier.

10. The apparatus of claim 1 wherein the means for classifying comprises a processor configured with instructions to execute a Support Vector Machine classifier on the multi-spectral channel values per pixel.

11. The apparatus of claim 10 wherein the means for classifying further comprises a processor configured to execute a vector quantizer on the multi-spectral channel values per pixel.

12. The apparatus of claim 1 wherein the means for classifying comprises a processor configured with instructions to execute a neural network classifier on the multi-spectral channel values per pixel.

13. The apparatus of claim 12 wherein the means for classifying comprises a processor configured with instructions to execute the neural network classifier on the multi-spectral channel values per pixel, wherein the spatial relationship function comprises 2D spatial information associated with the multi-spectral channel values per pixel.

14. The apparatus of claim 1 wherein the means for computing a spatial relationship function comprises a processor configured with instructions to determine a texture measure for a region of pixels in an image of the object.

15. The apparatus of claim 14 wherein the texture measure comprises pixel differences in the region.

16. The apparatus of claim 14 wherein the texture measure comprises a histogram of gradients.

17. The apparatus of claim 14 wherein the means for computing a spatial relationship function comprises a filter that compares pixel values of pixels in the region.

18. The apparatus of claim 14 wherein the means for computing a spatial relationship function comprises a processor configured with instructions to compute pixel differences of pixel values at pixel locations within the region, the pixel locations being at predetermined spacing and direction within the region.

19. The apparatus of claim 1 wherein the means for computing a spatial relationship function comprises a processor configured with instructions to determine a texture measure for a region of pixels in an image of the object;

and

the means for classifying comprises a processor configured with instructions to execute a vector quantizer on the multi-spectral channel values per pixel.

20. The apparatus of claim 19 wherein the means for classifying further comprises the processor configured with instructions to execute a Support Vector Machine classifier.