Method and apparatus for correcting crosstalk and spatial resolution for multichannel imaging

• US 20040161165A1
• Filed: 02/20/2004
• Published: 08/19/2004
• Est. Priority Date: 04/25/2001
• Status: Active Grant

• Current Assignee: Cytek Biosciences Inc
• Original Assignee: Amnis Corporation (DiaSorin S.p.A.)
• Inventors: Frost, Keith L., Riley, James K.
• Granted Patent: US 7,006,710 B2
• Time in Patent Office: Days
• US Class Current: 382/294
• CPC Class Codes: G01J 3/28 Investigating the spectrum using colour filters G01J3/51
• CPC Class Codes: G01J 3/2803 using photoelectric array detector
• CPC Class Codes: G01J 3/32 Investigating bands of a spectrum in sequence by a single detector
• CPC Class Codes: G01J 3/36 Investigating two or more bands of a spectrum by separate detectors
• CPC Class Codes: G01N 15/1433 using image recognition
• CPC Class Codes: G01N 15/147 the analysis being performed on a sample stream
• CPC Class Codes: G06F 2218/02 Preprocessing
• CPC Class Codes: G06T 2207/10036 Multispectral image; Hyperspectral image
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/32 using correlation-based methods
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 20/69 Microscopic objects, e.g. biological cells or cellular parts

Abstract

A multichannel imaging system generates an ensemble of images for each field of view of an object. Each image in the ensemble is intended to contain information from only one source among a plurality of sources for the object. However, due to crosstalk, at least a portion of the signal from a first source appears in a channel intended for a second source. Because the accuracy of the correction will be degraded if the images in an ensemble are spatially misaligned with respect to one another, the spatial offset between images is determined and a correction is applied to substantially eliminate the offset. Then, a correction to the signals is determined to substantially reduce the contributions to the signal in a channel from the signals in other channels. The signal processing can be employed to process the output signals for each of a plurality of different disclosed imaging systems.

Claims

1. A method for processing signals from different channels in a multi-channel digital imaging system, said signals comprising image signal data, comprising steps of:

(a) determining spatial alignment offsets for the image signal data;

(b) aligning the image signal data by applying the spatial alignment offsets to the image signal data, to produce aligned image signal data;

(c) determining spectral crosstalk reduction coefficients; and

(d) applying the spectral crosstalk coefficients to the aligned image signal data, for spectral correction thereof.

2. The method of claim 1, wherein the step of determining spatial alignment offsets comprises the step of determining spatial alignment offsets to within sub-pixel resolution, and wherein the step of aligning the image signal data comprises the step of aligning the image signal data by applying the sub-pixel spatial alignment offsets to the image signal data.

3. The method of claim 1, wherein the step of determining spatial alignment offsets is performed before sample data are captured by the multi-channel digital imaging system.

4. The method of claim 1, wherein the step of determining spatial alignment offsets is performed concurrently with the step of capturing sample data with the multi-channel digital imaging system.

5. The method of claim 1, wherein the step of determining spatial alignment offsets comprises the steps of:

(a) selecting a first one of said different channels as a reference channel;

(b) selecting a different one of said different channels as a data channel, image signal data in the data channel to be aligned with image signal data in the reference channel;

(c) producing a correlogram by processing image signal data from the reference channel with image signal data from the data channel;

(d) determining a peak of the correlogram; and

(e) comparing the peak of correlogram with image signal data in the reference channel to determine the spatial alignment offsets.

6. The method of claim 5, wherein the step of producing a correlogram comprises the step of processing image signal data from the reference channel with image signal data from the data channel in the frequency domain.

7. The method of claim 5, wherein the step of producing a correlogram comprises the step of processing image signal data from the reference channel with image signal data from the data channel in the spatial domain.

8. The method of claim 5, wherein the step of producing a correlogram comprises the step of processing a subset of image signal data from the reference channel with a subset of image signal data from the data channel, wherein the possible spatial alignment offsets correspond to the subsets.

9. The method of claim 5, wherein the step of preparing a correlogram comprises the step of using boundary data corresponding to image signal data the reference channel and image signal data in the data channel.

10. The method of claim 9, wherein the boundary data are generated using a two dimensional gradient operator.

11. The method of claim 1, wherein the step of aligning the image signal data comprises the step of convolving the image signal data using an interpolation kernel to enable image signal data to be aligned with sub-pixel resolution.

12. The method of claim 11, wherein the interpolation kernel is determined using a function approximation of the peak of a correlogram using a Taylor series expansion.

13. The method of claim 1, wherein the step of determining spectral crosstalk reduction coefficients comprises the step of imaging a control sample, wherein said control sample comprises a source substantially limited to a single one of the different channels.

14. The method of claim 13, wherein the control sample comprises at least one of a synthesized bead and a biological sample.

15. The method of claim 1, wherein the step of determining spectral crosstalk reduction coefficients comprises the step of employing a theoretical model of a crosstalk spectrum and a sensitivity of a camera used to capture an image of a sample to a stimulus of a spectrum, in producing the image signal data.

16. The method of claim 1, wherein the step of determining spectral crosstalk reduction coefficients comprises the step of solving linear equations.

17. The method of claim 16, wherein the step of solving linear equations comprises the step of utilizing a singular value decomposition.

18. The method of claim 1, wherein the step of applying the spectral crosstalk coefficients for spectral correction comprises the step of applying a linear equation.

19. A method for reducing crosstalk among a plurality of signals from a plurality of sources, each signal being assigned to a separate channel and primarily containing information corresponding to a different source among the plurality of sources, comprising the steps of:

(a) applying spatial corrections to correct any misalignment of the signals between channels, such that corresponding signals from different sources in the plurality of channels are aligned; and

(b) for each channel, substantially reducing erroneous contributions to the signal assigned to the channel from others of the plurality of signals.

20. The method of claim 19, wherein the step of applying spatial corrections to correct any misalignment of the signals between channels comprises the step of applying spatial corrections at a sub-pixel resolution.

21. A method for correcting errors in a multichannel signal, the multichannel signal comprising an ensemble of related signals used to produce images for different channels, wherein each signal is associated with a different channel and is intended to provide information from only one source, comprising the steps of:

(a) aligning the signals in the ensemble relative to each other, such that when images produced from the signals are displayed, each image produced from a signal in the ensemble is substantially aligned with images produced from other signals in the ensemble;

(b) determining spectral crosstalk corrections suitable for correcting channel-to-channel crosstalk between signals of the ensemble; and

(c) applying the spectral crosstalk corrections to the signals associated with the different channels, to correct for the channel-to-channel crosstalk between the signals.

22. The method of claim 21, wherein the step of aligning comprises the step of applying:

(a) horizontal and vertical spatial offsets derived from a calibration signal; and

(b) constants that are accessed during the step of aligning, but which are not modified.

23. The method of claim 21, wherein the step of aligning comprises the step of applying spatial corrections at a sub-pixel resolution.

24. The method of claim 21, further comprising the step of providing a calibration signal, wherein the step of aligning comprises the step of generating horizontal and vertical spatial offsets based upon a comparison of each signal in the ensemble with the calibration signal.

25. The method of claim 24, wherein the step of providing a calibration signal comprises the step of imaging a control sample having a single source.

26. The method of claim 24, wherein the step of providing a calibration signal further comprising the step of providing the calibration signal when the multichannel system is initialized.

27. The method of claim 24, wherein the step of providing a calibration signal further comprising the step of providing the calibration signal periodically during the use of the multichannel system.

28. The method of claim 24, wherein each signal in the ensemble comprises image data, and horizontal and vertical spatial offsets are determined for each pixel of the image data, to align the images in the different channels.

29. The method of claim 28, wherein the step of aligning, for successive signals in the ensemble that are processed, comprises the steps of:

(a) detecting a boundary of a signal currently being processed;

(b) preparing a correlogram based on the boundary and a reference signal, thereby enabling location of a peak in the correlogram;

(c) repositioning the signal currently being processed, to correspond to the peak of the correlogram.

30. The method of claim 29, wherein the step of detecting a boundary of the signal currently being processed comprises the step of using a two-dimensional gradient operator to suppress flat surfaces and to enhance object boundaries.

31. The method of claim 29, wherein the step of preparing a correlogram based on the boundary and the reference signal comprises the step of preparing a correlogram in a frequency domain.

32. The method of claim 31, wherein the step of preparing a correlogram in the frequency domain comprises the steps of:

(a) performing a Fourier Transform of boundary data for a selected signal from the ensemble and a Fourier Transform of the reference signal;

(b) multiplying a result of the Fourier Transform of the boundary data for the selected signal by a result of the Fourier Transform of the reference signal to generate a product; and

(c) performing an inverse Fourier Transform of the product.

33. The method of claim 29, wherein the step of preparing a correlogram based on the boundary and the reference signal comprises the step of preparing a correlogram in the spatial domain.

34. The method of claim 33, wherein the step of preparing a correlogram in the spatial domain comprises the steps of performing signal processing upon a subset of possible spatial alignment offsets.

35. The method of claim 29, wherein groups of image data in each channel of the multichannel system are processed together, such that a cumulative correlogram is generated for each successive channel that is processed.

36. The method of claim 29, wherein the step of aligning further comprises the step of reconstructing each signal in the ensemble by interpolating a position of the image produced with the signal to a fraction of a pixel.

37. The method of claim 36, wherein the step of reconstructing each signal comprises the step of applying a two-dimensional interpolation.

38. The method of claim 37, wherein the step of applying a two-dimensional interpolation comprises the step of computing a new amplitude value for each pixel based on a weighted sum of a group of surrounding pixels.

39. The method of claim 38, wherein the step of computing a new amplitude value for each pixel is based on a weighted sum of a group of nine pixels, eight pixels of which surround an origin pixel.

40. The method of claim 38, wherein the step of computing a new amplitude value for each pixel comprises the step of applying a matrix of coefficients to each pixel value, wherein a sum the of the coefficients in the matrix is equal to 1.0.

41. The method of claim 29, further comprising the step of determining the peak of the correlogram by employing a Taylor series expansion, eigenvalues, and eigenvectors.

42. The method of claim 22, wherein the second class of constants comprises at least one of channel start columns for each signal, and inverted source coefficients.

43. The method of claim 22, wherein the step of applying the spectral crosstalk corrections comprises the step of employing constants that are accessed, but which are not modified.

44. The method of claim 21, wherein the step of aligning comprises the step of aligning the signal in real time.

45. The method of claim 24, further comprising the step of storing the signals for a period of time, wherein the step of applying spectral crosstalk corrections comprises the step of applying spectral crosstalk corrections to at least one of the signals that have been stored for the period of time.

46. The method of claim 21, further comprising the step of storing the signals for a period of time, wherein the step of aligning the signals in the ensemble comprises the step of aligning signals that have been stored for the period of time.

47. A method for correcting errors in a multichannel imaging system, wherein each channel has signal information relating to an image of an object, comprising the steps of:

(a) focusing light from the object along a collection path;

(b) dispersing the light that is traveling along the collection path into a plurality of light beams, such that each light beam corresponds to a different source on the object;

(c) focusing each of the light beams to produce respective images for the light beams;

(d) providing at least one detector disposed to receive the respective images and in response, generating an output signal corresponding to each image;

(e) correcting misalignment of the images on said at least one detector to within sub-pixel resolution, so that all of the output signals are substantially aligned in time; and

(f) substantially reducing crosstalk contributions to each output signal from other output signals.

48. A method for correcting errors in a multichannel imaging system, wherein each channel is intended to contain signal information relating to an image of an object, comprising the steps of:

(a) focusing light from the object along a collection path;

(b) dispersing the light that is traveling along the collection path into a plurality of light beams, such that each light beam corresponds to a different source;

(c) focusing each of the light beams to produce respective images for the light beams;

(d) providing at least one detector disposed to receive the respective images and in response, generating an output signal corresponding to each image;

(e) correcting misalignment of the images on said at least one detector, so that all of the output signals are substantially aligned in time; and

(f) substantially reducing crosstalk contributions to each output signal from other output signals.

49. The method of claim 48, wherein the step of correcting misalignment comprises the steps of:

(a) determining a spatial offset between an image currently being processed and a reference image; and

(b) applying a correction factor to the output signal of the image currently being processed to substantially eliminate the spatial offset.

50. The method of claim 49, wherein the step of applying a correction factor so as to substantially eliminate the spatial offset comprises the steps of:

(a) spatially adjusting image data in the output signal so that the image data in the output signal is aligned with the reference image data to the nearest pixel; and

(b) reconstructing the output signal by interpolating a remainder of the spatial offset to a fraction of a pixel, to further reduce the spatial offset in the output signal for the image currently being processed.

51. The method of claim 48, wherein the step of substantially reducing crosstalk contributions to the output signal comprises the step of solving a set of linear equations, wherein each output signal is represented by a linear equation.

52. The method of claim 51, wherein the step of solving a set of linear equations comprises the step of solving the set of linear equations for each pixel in the image that produces the output signal.

53. The method of claim 52, wherein the step of solving a set of linear equations comprises the step of applying a singular value decomposition to a matrix form of the set of linear equations.

54. A multichannel imaging system for generating an ensemble of images from an object for each field of view of the object, wherein each image in the ensemble contains information from substantially only one source, comprising:

(a) a collection lens disposed so that light traveling from the object passes through the collection lens and travels along a collection path;

(b) a dispersing component disposed in the collection path so as to receive the light that has passed through the collection lens, dispersing the light into a plurality of light beams, each light beam being directed away from the dispersing component in a different predetermined direction;

(c) an imaging lens disposed to receive the light beams from the dispersing component, thereby producing said ensemble of images, each image being produced from a different one of the light beams and being projected by the imaging lens toward a different predetermined location;

(d) a multichannel detector disposed to receive the plurality of images produced by the imaging lens, the multichannel detector producing a plurality of output signals, such that a separate output signal is produced for each of said light beams; and

(e) means for processing the output signals to;

(i) correct the output signals for any misalignment between the images in the ensemble on the multichannel detector to within sub-pixel resolution; and

(ii) reduce contributions from the output signals of other channels, to the output signal in each channel.

55. A multichannel imaging system for generating an ensemble of images from an object having a plurality of field of views, for each field of view of the object, wherein each image in the ensemble contains information from substantially only one source, comprising:

(a) a collection lens disposed so that light traveling from the object passes through the collection lens and travels along a collection path;

(b) a dispersing component disposed in the collection path so as to receive the light that has passed through the collection lens, dispersing the light into a plurality of light beams, each light beam being directed away from the dispersing component in a different predetermined direction;

(c) an imaging lens disposed to receive the light beams from the dispersing component, thereby producing said ensemble of images, each image being produced from a different one of the light beams and being projected by the imaging lens toward a different predetermined location;

(d) a multichannel detector disposed to receive the plurality of images produced by the imaging lens, the multichannel detector producing a plurality of output signals, such that a separate output signal is produced for each of said light beams; and

(e) means for processing the output signals to;

(i) correct the output signals for any misalignment between the images in the ensemble on the multichannel detector; and

(ii) reduce contributions from the output signals of other channels, to the output signal in each channel.

56. The system of claim 55, further comprising a display electrically coupled to said means for processing, said display producing an image in response to the output signal as processed by said means for processing.

57. The system of claim 55, wherein said means for processing comprises:

(a) a memory in which a plurality of machine instructions defining a signal conditioning function are stored; and

(b) a processor that is coupled to the memory to access the machine instructions, said processor executing said machine instructions and thereby implementing a plurality of functions, including;

(i) processing the output signals to spatially align the images detected by the multichannel detector; and

(ii) applying a spectral crosstalk correction to the output signals, to remove a channel-to-channel crosstalk.

58. The system of claim 55, wherein said multichannel detector comprises a time delay integration (TDI) detector, said TDI detector producing said output signals by integrating light from at least a portion of the object over time, while a relative movement between the object and the imaging system occurs.

59. A multichannel imaging system for generating an ensemble of images of an object, wherein each image in the ensemble contains information from substantially only a single source from among a plurality of sources, comprising:

(a) a first collection lens disposed so that light from the object passes through the first collection lens and travels along a first collection path;

(b) a first light dispersing element disposed in the first collection path so as to disperse the light that has passed through the first collection lens, producing a first source from among said plurality of sources;

(c) a first imaging lens disposed to receive light from the first source, producing a first image from the first source, said first image being a first one of said ensemble of images;

(d) a first detector disposed to receive said first image produced by the first imaging lens, and in response thereto, producing a first output signal;

(e) a second collection lens disposed so that light from the object passes through the second collection lens and travels along a second collection path different than the first collection path;

(f) a second light dispersing element disposed in the second collection path so as to disperse the light that has passed through the second collection lens, producing a second source from among said plurality of sources;

(g) a second imaging lens disposed to receive light from the second source, producing a second image from the second source, said second image comprising a second one of said ensemble of images;

(h) a second detector disposed to receive said second image produced by the second imaging lens, producing a second output signal; and

(i) means coupled to each detector, for processing the first and the second output signals to perform the following functions;

(i) correcting the output signals for misalignment between the images on the first and the second detector; and

(ii) substantially reducing contributions to each of the first and the second output signals from the other of the first and the second output signals.

60. The system of claim 59, further comprising a display electrically coupled to said means for processing, said display reproducing an image in response to each output signal as processed by said means for processing.

61. The system of claim 59, wherein said means processing comprises an oscilloscope.

62. The system of claim 59, wherein said means for processing comprises a programmed computer.

63. The system of claim 59, wherein said means for processing comprises an application specific integrated circuit.

64. The system of claim 59, wherein each detector comprises a pixilated detector.

65. A multichannel imaging system for generating an ensemble of images of an object, wherein each image in the ensemble contains information from substantially only a single source from among a plurality of sources, comprising:

(a) a collection lens disposed so that light traveling from the object passes through the collection lens and is focussed along a collection path;

(b) a dispersing component that receives the light from the collection lens and disperses the light into a plurality of light beams, as a function of a plurality of different discriminable characteristics of the light, each of said plurality of light beams corresponding to a different one of said plurality of sources;

(c) at least one pixilated detector;

(d) an imaging lens that focuses each of the plurality of light beams on said at least one pixilated detector, producing a respective image corresponding to a different one of the plurality of light beams, each image being one of said ensemble of images, said at least one pixilated detector providing an output signal for each respective image; and

(e) a signal processor coupled to receive the output signals from said at least one pixilated detector, said signal processor processing the output signals to;

(i) correct the output signals for any misalignment between the respective images on said at least one pixilated detector; and

(ii) substantially reducing crosstalk between the output signals.

66. The system of claim 65, wherein said pixilated detector comprises a time delay integration (TDI) detector, said TDI detector produces said output signals by integrating light from at least a portion of the object over time, while a relative movement between the object and the imaging system occurs.

67. The system of claim 65, wherein said signal processor comprises an oscilloscope.

68. The system of claim 65, wherein said signal processor comprises a programmed computer.

69. The system of claim 65, wherein said signal processor comprises an application specific integrated circuit.

70. An article of manufacture adapted for use with a computer, comprising:

(a) a memory medium; and

(b) a plurality of machine instructions, which are stored on the memory medium, said plurality of machine instructions when executed by a computer, causing the computer to;

(i) correct a signal misalignment between a set of related signals to within sub-pixel resolution, wherein each one of the set of related signals primarily contains information corresponding to a different specific source; and

(ii) substantially reduce crosstalk contributions to each of the signals from other of the signals in the set of related signals.

71. An article of manufacture adapted for use with a processor, comprising:

(a) a memory medium; and

(b) a plurality of machine instructions, which are stored on the memory medium, said plurality of machine instructions when executed by a processor, causing the processor to;

(i) correct a signal misalignment between a set of related signals, wherein each one of the set of related signals primarily contains information corresponding to a different specific source; and

(ii) substantially reduce crosstalk contributions to each of the signals from other of the signals in the set of related signals.

72. A system for reducing crosstalk among a plurality of related signals in a set, each one of the set of related signals primarily containing information corresponding to a specific different source from among a plurality of different sources, comprising:

(a) a memory in which a plurality of machine instructions defining the parent application are stored; and

(b) a processor that is coupled to the memory to access the machine instructions, said processor executing said machine instructions and thereby implementing a plurality of functions, including;

(i) correcting a signal misalignment between the plurality of related signals, each one of the plurality of related signals in the set is substantially aligned with other of the plurality of related signals in the set; and

(ii) for each one of the plurality of related signals in the set, reducing crosstalk contributions from other of the plurality of related signals.