Method for classifying a pixel of a hyperspectral image in a remote sensing application

• US 9,019,358 B2
• Filed: 02/08/2011
• Issued: 04/28/2015
• Est. Priority Date: 02/08/2011
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Mestha, Lalit Keshav, Xu, Beilei, Paul, Peter
• Primary Examiner(s): Dastouri, Mehrdad
• Assistant Examiner(s): Anderson, II, James
• Application Number:
  US13/023,310
• Publication Number:
  US 20120200682A1
• Time in Patent Office: 1,540 Days
• Field of Search: None
• US Class Current: 348/61
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 20/194 using hyperspectral data, i.e. more or other wavelengths than RGB

Abstract

What is disclosed is a novel system and method for simultaneous spectral decomposition suitable for image object identification and categorization for scenes and objects under analysis. The present system captures different spectral planes simultaneously using a Fabry-Perot multi-filter grid each tuned to a specific wavelength. A method for classifying pixels in the captured image is provided. The present system and method finds its uses in a wide array of applications such as, for example, occupancy detection in a transportation management system and in medical imaging and diagnosis for healthcare management. The teachings hereof further find their uses in other applications where there is a need to capture a two dimensional view of a scene and decompose the scene into its spectral bands such that objects in the image can be appropriately identified.

Claims

1. A method for identifying a target object in a hyperspectral image, the method comprising:

capturing different spectral planes of an image using a hyperspectral camera having a multi-filter grid, the multi-filter grid comprising a plurality of filters, each filter being tuned to a respective band comprising a respective specified wavelength of the electromagnetic spectrum, each respective band corresponding to a respective spectral plane;

constructing a hyperspectral image data cube comprising a plurality of 2-dimensional arrays of pixels, each 2-dimensional array comprising respective intensity values for each pixel of a respective band corresponding to a respective spectral plane; and

for each pixel location in said hyperspectral image data cube;

constructing at least one multi-dimensional vector comprising intensity values of at least one pixel from across at least a portion of said spectral planes, each dimension of said vectors corresponding to a respective intensity of a respective spectral plane;

calculating a magnitude of said pixel'"'"'s multi-dimensional vector;

comparing said magnitude to a corresponding magnitude calculated for a reference vector predetermined for an identified known sample;

determining whether said pixel belongs to said known sample based upon a determination by said comparing that said magnitude is within a threshold value of said corresponding magnitude calculated for said reference vector; and

classifying said pixel based upon said determination.

2. The method of claim 1, wherein said comparing said magnitude comprises:

forming a ratio of said magnitude of said pixel'"'"'s multi-dimensional vector and said corresponding magnitude calculated for said reference vector; and

obtaining a numeric value result from said ratio,

wherein said determination by said comparing that said magnitude is within a threshold value of said corresponding magnitude calculated for said reference vector is based upon said numeric value.

3. The method of claim 1, further comprising pre-processing said images in advance of constructing said hyperspectral image data cube, said pre-processing comprising any of:

improving a contrast of said image;

correcting said image for skew;

magnifying any of said image;

performing a frame boundary detection on said image;

stitching images together;

cropping said image into frames;

performing a frame-to-frame registration on said image;

re-scaling any intensity values associated with said image based on the sensor response and an illumination at each wavelength; and

selecting an area of interest in any of said images.

4. The method of claim 1, wherein said known sample comprises human skin.

5. The method of claim 1, further comprising using said classified pixels to determine a number of live objects in any of said captured hyperspectral images.

6. The method of claim 1, wherein said multi-filter grid comprises any of:

a Fabry-Perot filter for simultaneous multi-band capture of said different planes, and a filter wheel for non-simultaneous multi-band capture of said different planes.

7. The method of claim 1, further comprising:

grouping, prior to processing said vectors, similar vectors from said hyperspectral image data cube using a clustering algorithm; and

determining a representative vector representing a cluster center of said similar vectors,

wherein the processing said vectors comprises comparing said representative vector to said reference vector, and

wherein the classifying comprises determining whether all of said similar vectors belongs to said known sample based upon the comparing said representative vector to said reference vector.

22. The method of claim 1, wherein the plurality of filters comprises filters with respective wavelengths between 1400 nm and 1800 nm.

23. The method of claim 22, wherein the plurality of filters comprises filters with respective wavelengths covering an optical spectrum between at least 1500 nm and 1750 nm.

8. A hyperspectral imaging system for identifying a target object, the system comprising:

a hyperspectral camera that captures images simultaneously, said camera having an array of sensors and a multi-filter grid for multi-band capture, the multi-filter grid comprising a plurality of filters, each filter being tuned to a respective band comprising a respective specified wavelength of the electromagnetic spectrum, each respective band corresponding to a respective spectral plane;

a memory and a storage medium; and

a processor in communication with and said storage medium, said memory, and said camera, said processor executing machine readable instructions for performing the method of;

capturing different spectral planes of an image using said hyperspectral camera;

constructing a hyperspectral image data cube comprising a plurality of 2-dimensional arrays of pixels, each 2-dimensional array comprising respective intensity values for each pixel of a respective band corresponding to a respective spectral plane; and

for each pixel location in said hyperspectral image data cube;

constructing at least one multi-dimensional vector comprising intensity values of at least one pixel from across at least a portion of said spectral planes, each dimension of said vectors corresponding to a respective intensity of a respective spectral plane;

calculating a magnitude of said pixel'"'"'s multi-dimensional vector;

comparing said magnitude to a corresponding magnitude calculated for a reference vector predetermined for an identified known sample;

determining whether said pixel belongs to said known sample based upon a determination by said comparing that said magnitude is within a threshold value of said corresponding magnitude calculated for said reference vector; and

classifying said pixel based upon said determination.

9. The system of claim 8, wherein said comparing said magnitude comprises:

forming a ratio of said magnitude of said pixel'"'"'s multi-dimensional vector and said corresponding magnitude calculated for said reference vector; and

obtaining a numeric value result from said ratio,

wherein said determination by said comparing that said magnitude is within a threshold value of said corresponding magnitude calculated for said reference vector is based upon said numeric value.

10. The system of claim 8, further comprising pre-processing said images in advance of constructing said hyperspectral image data cube, said pre-processing comprising any of:

improving a contrast of said image;

correcting said image for skew;

magnifying any of said image;

performing a frame boundary detection on said image;

stitching images together;

cropping said image into frames;

performing a frame-to-frame registration on said image;

re-scaling any intensity values associated with said image based on the sensor response and illumination at each wavelength; and

selecting an area of interest in any of said images.

11. The system of claim 8, wherein said known sample comprises human skin.

12. The system of claim 8, further comprising using said classified pixels to determine a number of live objects in any of said captured hyperspectral images.

13. The system of claim 8, wherein said multi-filter grid comprises any of:

a Fabry-Perot filter for simultaneous multi-band capture of said different planes, and a filter wheel for non-simultaneous multi-band capture of said different planes.

14. The system of claim 8, further comprising:

grouping, prior to processing said vectors, similar vectors from said hyperspectral image data cube using a clustering algorithm; and

determining a representative vector representing a cluster center of said similar vectors,

wherein the processing said vectors comprises comparing said representative vector to said reference vector, and

wherein the classifying comprises determining whether all of said similar vectors belongs to said known sample based upon the comparing said representative vector to said reference vector.

15. A computer implemented method for identifying a target object in a hyperspectral image, the method comprising:

capturing different spectral planes of an image using a hyperspectral camera having a multi-filter grid, the multi-filter grid comprising a plurality of filters, each filter being tuned to a respective band comprising a respective specified wavelength of the electromagnetic spectrum, each respective band corresponding to a respective spectral plane;

pre-processing said images;

constructing a hyperspectral image data cube comprising a plurality of 2-dimensional arrays of pixels, each 2-dimensional array comprising respective intensity values for each pixel of a respective band corresponding to a respective spectral plane; and

for each pixel location in said hyperspectral image data cube;

constructing at least one multi-dimensional vector comprising intensity values of at least one pixel from across at least a portion of said spectral planes, each dimension of said vectors corresponding to a respective intensity of a respective spectral plane;

calculating a magnitude of said pixel'"'"'s multi-dimensional vector;

comparing said magnitude to a corresponding magnitude calculated for a reference vector predetermined for an identified known sample;

determining whether said pixel belongs to said known sample based upon a determination by said comparing that said magnitude is within a threshold value of said corresponding magnitude calculated for said reference vector; and

classifying said pixel based upon said determination.

16. The computer implemented method of claim 15, wherein said comparing said magnitude comprises:

forming a ratio of said magnitude of said pixel'"'"'s multi-dimensional vector and said corresponding magnitude calculated for said reference vector; and

obtaining a numeric value result from said ratio,

wherein said determination by said comparing that said magnitude is within a threshold value of said corresponding magnitude calculated for said reference vector is based upon said numeric value.

17. The computer implemented method of claim 15, wherein pre-processing said images comprises any of:

improving a contrast of said image;

correcting said image for skew;

magnifying any of said image;

performing a frame boundary detection on said image;

stitching images together;

cropping said image into frames;

performing a frame-to-frame registration on said image;

re-scaling any intensity values associated with said image based on the sensor response and illumination at each wavelength;

shifting any of said image; and

selecting an area of interest in any of said images.

18. The computer implemented method of claim 15, wherein said known sample comprises human skin.

19. The computer implemented method of claim 15, further comprising using said classified pixels to determine a number of live objects in any of said captured hyperspectral images.

20. The computer implemented method of claim 15, wherein said multi-filter grid comprises any of:

a Fabry-Perot filter for simultaneous multi-band capture of said different planes, and a filter wheel for non-simultaneous multi-band capture of said different planes.

21. The computer implemented method of claim 15, further comprising:

grouping, prior to processing said vectors, similar vectors from said hyperspectral image data cube using a clustering algorithm; and

determining a representative vector representing a cluster center of said similar vectors,

wherein the processing said vectors comprises comparing said representative vector to said reference vector, and

wherein the classifying comprises determining whether all of said similar vectors belongs to said known sample based upon the comparing said representative vector to said reference vector.