Methods and systems for correcting image misalignment

• US 7,187,810 B2
• Filed: 10/18/2002
• Issued: 03/06/2007
• Est. Priority Date: 12/15/1999
• Status: Expired due to Fees

• Current Assignee: MediSpectra, Inc. (SpectraScience, Inc.)
• Original Assignee: MediSpectra, Inc. (SpectraScience, Inc.)
• Inventors: Jiang, Chunsheng, Clune, Thomas, Schmid, Philippe
• Primary Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US10/273,511
• Publication Number:
  US 20030095721A1
• Time in Patent Office: 1,600 Days
• Field of Search: 382128-134, 382/289, 382/294, 382/260, 382/278, 600/310, 600/410, 600/443, 324/309
• US Class Current: 382/294
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 1/0008 characterised by distal tip features
• CPC Class Codes: A61B 1/00142 with means for preventing contamination, e.g. by using a sanitary sheath
• CPC Class Codes: A61B 1/043 for fluorescence imaging
• CPC Class Codes: A61B 5/0071 by measuring fluorescence emission
• CPC Class Codes: A61B 5/0084 for introduction into the body, e.g. by catheters A61B5/1459 takes precedence
• CPC Class Codes: A61B 5/7257 using Fourier transforms
• CPC Class Codes: A61M 11/00 Sprayers or atomisers specially adapted for therapeutic purposes in general B05B; aerosol containers B65D83/14
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 7/262 using transform domain methods, e.g. Fourier domain methods
• CPC Class Codes: G06T 7/37 using transform domain methods

Abstract

The invention provides methods of determining a correction for a misalignment between at least two images in a sequence of images due at least in part to sample movement. The methods are applied, for example, in the processing and analysis of a sequence of images of biological tissue in a diagnostic procedure. The invention also provides methods of validating the correction for a misalignment between at least two images in a sequence of images of a sample. The methods may be applied in deciding whether a correction for misalignment accurately accounts for sample motion.

Claims

1. A method of compensating for image misalignment, the method comprising the steps of:

obtaining a set of sequential images of a tissue sample; and

correcting for a misalignment between each of a pluarlity of pairs of the sequential images, the misalignment due at least in part to movement of the tissue sample, wherein the correcting step comprises determining a correction for a misalignment between a pair of the sequential images less than about 2 seconds after obtaining the latter of the pair of images.

2. The method of claim 1, wherein the correcting step is performed in real time.

3. The method of claim 1, wherein the correcting step comprises adjusting an optical signal detection device used to obtain the set of sequential images.

4. The method of claim 3, wherein the correcting step comprises adjusting a position of a component of the optical signal detection device.

5. The method of claim 4, wherein the component comprises a mirror.

6. The method of claim 1, wherein the tissue sample is an in-situ tissue sample and wherein the misalignment is due at least in part to patient motion.

7. The method of claim 1, further comprising the step of applying a contrast agent to the tissue sample.

8. The method of claim 1, wherein the correcting step comprises electronically adjusting at least one of the sequential images.

9. The method of claim 1, wherein the pair of images are consecutive images.

10. The method of claim 1, wherein the correcting step comprises the step of filtering a subset of data from at least one of the sequential images.

11. The method of claim 10, wherein the correcting step comprises the step of preprocessing the subset of data prior to the filtering.

12. The method of claim 10, wherein the filtering step comprises at least one of frequency domain filtering and discrete convolution in the space domain.

13. The method of claim 10, wherein the filtering step comprises Laplacian of Gaussian filtering.

14. The method of claim 10, wherein the filtering step comprises using a feathering technique.

15. The method of claim 10, wherein the filtering step comprises using a Hamming window.

16. The method of claim 1, wherein the correcting step comprises computing a cross correlation using data from the pair of images.

17. The method of claim 16, wherein the computing of the cross correlation comprises computing a product represented by

F_i(u,v)F*_j(u,v),

where F_i(u,v) is a Fourier transform of data derived from a subset of data from a first image, i, of the pair of images, F*_j(u,v) is a complex conjugate of a Fourier transform of data derived from a subset of data from a second image, j, of the pair of images, and u and v are frequency domain variables.

18. The method of claim 17, wherein the computing of the cross correlation comprises computing an inverse Fourier transform of the product.

19. The method of claim 1, wherein the tissue sample comprises cervical tissue.

20. The method of claim 1, wherein the tissue sample comprises at least one member of the group consisting of colorectal tissue, gastroesophageal tissue, urinary bladder tissue, lung tissue, and skin tissue.

21. The method of claim 1, wherein the tissue sample comprises epithelial cells.

22. The method of claim 1, wherein the obtaining step comprises obtaining the set of sequential images of the tissue sample during application of a chemical agent to the tissue sample.

23. The method of claim 1, wherein the obtaining step comprises obtaining the set of sequential images of the tissue sample after application of a chemical agent to the tissue sample.

24. The method of claim 23, wherein the chemical agent comprises at least one member selected from the group consisting of acetic acid, formic acid, propionic acid, and butyric acid.

25. The method of claim 23, wherein the chemical agent is selected from the group consisting of Lugol'"'"'s iodine, Shiller'"'"'s iodine, methylene blue, toluidine blue, indigo carmine, indocyanine green, and fluorescein.

26. The method of claim 1, wherein the obtaining step comprises obtaining the set of sequential images of the tissue sample during an acetowhitening test.

27. The method of claim 1, wherein the movement of the tissue sample is relative to an optical signal detection device and comprises at least one member of the group consisting of translational motion, rotational motion, warping, and local deformation.

28. The method of claim 1, wherein one or more images of the set of sequential images comprise measurements of an optical signal from the tissue sample.

29. The method of claim 28, wherein the optical signal comprises visible light.

30. The method of claim 28, wherein the optical signal comprises fluorescent light.

31. The method of claim 28, wherein the optical signal is emitted by the tissue sample.

32. The method of claim 28, wherein the optical signal is reflected by the tissue sample.

33. The method of claim 28, wherein the optical signal is transmitted through the tissue sample.

34. The method of claim 1 wherein the obtaining step and the correcting step are performed alternately.

35. The method of claim 1 wherein the obtaining step and the correcting step are performed substantially concurrently.

36. The method of claim 1 wherein the correcting step comprises determining the correction for the misalignment between the pair of the sequential images less than about one second after obtaining the latter of the pair of images.

37. The method of claim 1, further comprising the steps of:

adjusting one or both of the pair of images according to the correction;

following said adjusting step, defining one or more validation cells common to both of the pair of images;

for each of the one or more validation cells, computing a measure of displacement within the validation cell between the pair of images; and

validating the correction by comparing at least one of the measures of displacement with a threshold value.

38. The method of claim 37, wherein each of the one or more validation cells comprises a subset of a bounded image plane common to the pair of images.

39. The method of claim 38, wherein the one or more validation cells together comprise a central portion of the bounded image plane.

40. The method of claim 38, wherein the bounded image plane has an area about four times larger than a total area of the one or more validation cells.

41. The method of claim 37, wherein the validating step comprises eliminating from consideration one or more of the measures of displacement.

42. A system for compensating for image misalignment, the system comprising:

an optical signal detection device configured to obtain a set of sequential images of a tissue sample; and

a processor configured to determine a correction for a misalignment between each of a plurality of pairs of the sequential images, the misalignment due at least in part to movement of the tissue sample, wherein the correction between a pair of the sequential images is determined less than about 2 seconds after obtaining the latter of the pair of images.

43. The system of claim 42, wherein the optical signal detection device comprises a component that is adjusted according to the correction.

44. The system of claim 43, wherein the component comprises a mirror.

45. The system of claim 43, wherein the tissue sample is an in-situ tissue sample and wherein the misalignment is due at least in part to patient motion.

46. The system of claim 43, further comprising an applicator for applying a contrast agent to the tissue sample.

47. The system of claim 46, wherein the chemical agent comprises at least one member selected from the group consisting of acetic acid, formic acid, propionic acid, and butyric acid.

48. The system of claim 46, wherein the chemical agent comprises acetic acid.

49. A method of compensating for image misalignment, the method comprising the steps of:

obtaining a sequence of images of a tissue sample; and

correcting for misalignment between at least two of the images, said misalignment being due at least in part to movement of the tissue sample, wherein the correcting step comprises adjusting a position of a mirror of an optical signal detection device used to obtain the sequence of images.

50. A method of compensating for image misalignment, the method comprising the steps of:

obtaining a sequence of images of a tissue sample; and

correcting for misalignment between at least two of the images, said misalignment being due at least in part to movement of the tissue sample, wherein the correcting step comprises the step of filtering a subset of data from at least one image of the sequence of images using a feathering technique.

51. A method of compensating for image misalignment, the method comprising the steps of:

obtaining a sequence of images of a tissue sample; and

correcting for misalignment between at least two of the images, said misalignment being due at least in part to movement of the tissue sample, wherein the correcting step comprises the step of filtering a subset of data from at least one image of the sequence of images using a Hamming window.