Systems and methods for the multispectral imaging and characterization of skin tissue

• US 6,081,612 A
• Filed: 02/27/1998
• Issued: 06/27/2000
• Est. Priority Date: 02/28/1997
• Status: Expired due to Term

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University), STRATA Skin Sciences, Inc.
• Original Assignee: Electro-Optical Sciences, Inc.
• Inventors: Bogdan, Alexandru, Gutkowicz-Krusin, Dina, Jacobs, Adam, Elbaum, Marek, Greenebaum, Michael
• Primary Examiner(s): Chang, Jon
• Assistant Examiner(s): Dastouri, Mehrdad
• Application Number:
  US09/031,929
• Time in Patent Office: 851 Days
• Field of Search: 382/128, 382/132, 382/276, 382/232, 382/248, 382/240, 356/346, 348/398
• US Class Current: 382/128
• CPC Class Codes: A61B 5/0071 by measuring fluorescence emission
• CPC Class Codes: A61B 5/442 Evaluating skin mechanical properties, e.g. elasticity, hardness, texture, wrinkle assessment
• CPC Class Codes: A61B 5/444 Evaluating skin marks, e.g. mole, nevi, tumour, scar
• CPC Class Codes: A61B 5/445 Evaluating skin irritation or skin trauma, e.g. rash, eczema, wound, bed sore
• CPC Class Codes: A61B 5/726 using Wavelet transforms
• CPC Class Codes: G06T 2207/10064 Fluorescence image
• CPC Class Codes: G06T 2207/20064 Wavelet transform [DWT]
• CPC Class Codes: G06T 2207/30088 Skin; Dermal
• CPC Class Codes: G06T 2207/30096 Tumor; Lesion
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/42 using transform domain methods
• CPC Class Codes: G06T 7/90 Determination of colour characteristics

Abstract

Systems and methods for the multispectral imaging of skin tissue enables automatic characterization of the condition of a region of interest of the skin, based on direct digital imaging of the region of interest or the digitization of color photographic slides of the region of interest, illuminated by appropriately filtered light. Preferably, a digital image at a low spectral band is automatically segmented and that segmented mask is used to segment the other images by a digital processor. Parameters are estimated through wavelet maxima representations. Parameters related to the texture, asymmetry, blotchiness and border irregularities are also automatically estimated. The region of interest is automatically characterized by the digital processor, based on those parameters. The region of interest may include a skin lesion, in which case the present invention enables the characterization of the lesion as malignant or benign. The region of interest may also include wounds or burns.

Claims

1. A method of characterizing the condition of a region of interest of skin, wherein the absorption and scattering of light in different spectral bands by the region of interest is a function of the condition of the skin, the method comprising:

illuminating a portion of the skin including the region of interest by light in at least three spectral bands;

digitally imaging a portion of the skin including the region of interest at the at least three spectral bands with the light re-emitted by the portion of the skin to generate digital images comprising digital signals whose values are a function of the condition of the region of interest of the skin; and

providing the digital images to a processor, wherein the processor;

segments the digital images by generating a segmentation mask defining the boundary of the region of interest from a digital image in any one of the at least three spectral bands;

computes at least one rotationally and translationally invariant statistical measure of coefficient distributions of the multiscale wavelet maxima representations of the digital images in each spectral band, which are functions of the texture of the region of interest determined by the segmentation mask;

characterizes the condition of the skin based on the estimated values; and

outputs the characterization of the condition of the skin.

2. The method of claim 1, wherein the at least one statistical measure is calculated separately within either of a border region and an interior region of the digital image, wherein:

the border region encompasses the envelope of circles of fixed radius centered on the boundary of the segmentation mask; and

the inside region comprises all points of the image that are within the segmentation mask boundary but not included in the border region.

3. The method of claim 2, wherein the computing step comprises estimating at an individual level at least one value which is a statistical measure of texture of the portion of the region of interest within the border region and interior region, chosen from the group consisting of:

the number of wavelet maxima per unit area;

the ratio of the mean coefficient magnitude to the absolute deviation of the coefficient magnitudes from the mean value;

the ratio of the mean coefficient magnitude to the standard deviation of the coefficient magnitudes; and

the skewness of the coefficient magnitudes, normalized to the cube of the standard deviation.

4. The method of claim 1, further comprising estimating either of the degree of change of a statistic of the wavelet coefficient distribution with the increment of wavelet level, and the degree of deviation of such change from linearity.

5. The method of claim 2, further comprising estimating the average rate of change, with respect to level, of the number of wavelet maxima per unit area.

6. The method of claim 1, further comprising comparing the estimated texture values to the threshold derived from statistical analysis of a multiscale wavelet transformation of the digital image.

7. The method of claim 1, wherein the estimating and characterizing steps are conducted without the intervention of an operator.

8. The method of claim 1, wherein the segmenting step is conducted without the intervention of an operator.

9. The method of claim 1, wherein the illuminating step further comprises illuminating a region of interest including a burn.

10. The method of claim 9, wherein the characterizing step comprises characterizing the condition of the burn with respect to the condition of normal skin.

11. The method of claim 1, wherein the illuminating step further comprises illuminating a region of interest including a wound.

12. The method of claim 11, wherein the characterizing step comprises characterizing the condition of the wound with respect to the condition of normal skin.

13. The method of claim 1, further comprising:

photographing the region of interest with a color camera to form color photographic slides; and

illuminating the color photographic slides with light in each spectral band;

wherein the digital imaging step comprises digitally imaging the illuminated color photographic slides of the region of interest with a digital camera.

14. The method of claim 1, wherein the computing step further comprises estimating a value which is a function of the asymmetry of the segmented image in each spectral band, for two principal axis of the segmented image.

15. The method of claim 1, wherein the computing step further comprises computing the intensity moment with a binary intensity distribution.

16. The method of claim 1, wherein the computing step further comprises estimating at least one value which is a function of the blotchiness of the segmented digital image, the estimated blotchiness value being defined through statistical properties of the spatial distribution of topographic regions of the digital images at each spectral band.

17. The method of claim 16, wherein the computing step further comprises determining the centroids of topographic regions of the segmented digital image at each spectral band.

18. The method of claim 1, wherein the computing step comprises estimating a value which is a statistical measure of the deviation of the border of the region of interest from the border of an ellipse of the same area, aspect ratio, and orientation as the segmentation mask.

19. The method of claim 1, wherein the computing step comprises estimating a statistical measure of the gradient values of the intensity of the digital images across the border of the segmented images, at each spectral band.

20. The method of claim 1, wherein the computing step comprises estimating values based on the ratio of standard deviation of the areas of dermal papillae to their mean within the segmentation mask.

21. The method of claim 1, wherein the computing step comprises estimating values of the average and standard deviation of the thickness of rete ridges within the segmentation mask.

22. The method of claim 1, wherein the characterizing step comprises distinguishing multiple times between melanoma and several types of benign lesion.

23. The method of claim 1, wherein the condition of the region of interest to be characterized is the presence of a melanoma and the processor compares a weighted combination of parameter values against a threshold value for melanoma and different types of benign lesions, multiple times.

24. The method of claim 1, wherein the segmentation mask is generated from a digital image in a spectral band in which the amount of light re-emitted by skin is less within skin having an abnormal condition than within skin having a normal condition.

25. The method of claim 1, wherein the segmenting step further comprises segmenting the digital images by generating a segmentation mask in more than one spectral band.

26. The method of claim 1, wherein prior to the computing step, the digital image is first segmented to locate the region of interest in the image, and said computing step further comprises:

locating N topographic regions of the segmented image, wherein the nth such topographic regions is the locus of all pixels whose intensities line between I(n-1) and I(n), for n=1, 2, . . . , N; and

calculating at least one quantity that represents a statistical property of the spatial distribution of pixels within a topographic region.

27. The method of claim 26, wherein the at least one quantity is chosen from the group consisting of:

the means of the distances of all the pixels in a topographic region from the intensity centroid of the segmented image;

the standard deviation of said distances; and

the ratio of the area of a polygon defined by the intensity centroids of at least three different topographic regions to the total area of the segmented image.

28. The method of claim 1, wherein, in any spectral band, the digital image is segmented to locate the region of interest and the computing step further comprises:

locating the principal axes of computing an orientation angle in the segmented image;

rotating the segmented digital image such that the principal axes are parallel to the image axes;

computing the difference in intensity between each pair of pixels with locations that are mirror images with respect to a principal axis;

summing up the absolute values of intensity differences;

calculating asymmetry values for each principal axis by normalizing the sum of intensity differences by the total intensity in the segmented images; and

summing the estimated asymmetry values for the two principal axes.

29. The method of claim 28, wherein the digital image is the binary segmentation mask.

30. The method of claim 1, wherein, in any spectral band, the digital image is segmented to located the region of interest and the computing step further comprises:

locating N topographic regions of the segmented image, wherein the n^th such topographic region is the locus of all pixels whose intensities lie between I(n-1) and I(n), for n=1, 2, . . . , N; and

calculating at least one quantity that represents a statistical property of the spatial distribution of pixels within a topographic region.

31. The method of claim 30, wherein the calculated quantity depends on the mean and standard deviation of the distances of all the pixels in a topographic region from the intensity centroid of the segmented image.

32. The method of claim 31, wherein the calculated quantity is the ratio of the area of a polygon defined by the intensity centroids of at least three different topographic regions to the total area of the segmented image.

33. The method of claim 1, wherein the characterizing step comprises the logical union {`AND`} of the outputs of several intermediate distinguishing steps, wherein each such step distinguishes between a particular condition of a region of interest of the biological tissue and a different alternative condition of the same region of interest.

34. The method of claim 1, wherein the segmentation mask is generated from a digital image in that spectral band in which the resulting segmentation mask has the largest area.

35. A system for characterizing the condition of a region of interest of skin, comprising:

a source of illumination of light in at least three spectral bands;

a camera for acquiring digital images of the region of interest based on the light re-emitted from the illuminated region of interest at each of the spectral bands, the digital image comprising digital signals whose values are a function of the condition of the region of interest;

memory for storing the digital images provided by the camera;

a digital processor programmed to perform the steps of;

segmenting the digital images stored in memory by generating a segmentation mask from a digital image in any one of the at least three spectral bands;

computing at least one rotationally and translationally invariant statistical measure of coefficient distributions for the multiscale wavelet maxima representations of the digital images in each spectral band, which are functions of the texture of the region of interest determined by the segmentation mask;

characterizing the condition of the skin based on the estimated values;

outputting the characterization of the region of interest.

36. The system of claim 35, further comprising means for suppressing specular reflections from the region of interest.

37. The system of claim 36, wherein the processor estimates the statistical measures separately within either of a border region and an interior region of the digital image

the border region encompasses the envelope of circles of fixed radius centered on the boundary of the segmentation mask;

and

the inside region comprises all points of the image that are within the segmentation mask boundary but not included in the border region.

38. The system of claim 37, wherein the processor estimates at an individual level at least one value which is a statistical measure of texture of the portion of the region of interest within the border region and interior region, chosen from the group consisting of:

the number of wavelet maxima per unit area;

the ratio of the mean coefficient magnitude to the absolute deviation of the coefficient magnitudes from the mean value;

the ratio of the mean coefficient magnitude to the standard deviation of the coefficient magnitudes; and

the skewness of the coefficient magnitudes, normalized to the cube of the standard deviation.

39. The system of claim 38, wherein the processor further estimates either of the degree of change of a statistic of the wavelet coefficient distribution with increment of wavelet level, and the degree of deviation of such change from linearity.

40. The system of claim 38, wherein the processor further estimates the average rate of change, with respect to level, of the number of wavelet maxima per unit area.

41. The system of claim 35, wherein the camera records monochromatic images and the illumination means comprises:

a tungsten halogen light source with feedback to stabilize the intensity in each wavelength band;

means for sequentially filtering the light; and

an optical fiber ring illuminator to distribute the filtered light.

42. The system of claim 41, further comprising a feedback loop for stabilizing the intensity of the light source by the processor.

43. The system of claim 42, further comprising a material of stable reflectance for being illuminated by the light source, wherein the feedback loop includes the monitoring by the processor, of the intensity of light reflected from the material by the processor and the adjustment of the intensity of the light source if the monitored intensity varies from a desired value.

44. The system of claim 43, wherein the power, voltage or current supplied to the light source is monitored.

45. The system of claim 35, wherein the source of illumination is at least one laser.

46. A method of characterizing the condition of a region of interest of biological tissue, wherein the absorption and scattering of light in different spectral bands by the region of interest is a function of the condition of the tissue, the method comprising:

computing at least one rotationally and translationally invariant statistical measure of coefficient distributions of the multiscale wavelet maxima representations of at least three digital images of the region of interest, the at least three digital images formed by light of at least three wavelength bands, the statistical measure being a function of the texture of the region of interest; and

characterizing the condition of the tissue based on the generated measure.

47. The method of claim 46, wherein the at least one statistical measure is generated separately within one of either of a border region and an interior region of a feature of the image, wherein:

the border region encompasses the envelope of circles of fixed radius centered on the boundary of a segmentation mask of the feature of the image; and

the interior region comprises all points of the image that are within the segmentation mask boundary but not included in the border region.

48. The method of claim 47, wherein the generating step comprises generating at an individual level at least one value which is a statistical measure of texture of the portion of the region of interest within the border region and interior region, chosen from the group consisting of:

the number of wavelet maxima per unit area;

the ratio of the mean of the coefficient magnitudes to the absolute deviation of the coefficient magnitudes from the mean value;

the ratio of the mean of the coefficient magnitudes to the standard deviation of the coefficient magnitudes; and

the skewness of the coefficient magnitudes, normalized to the cube of the standard deviation of the coefficient magnitudes.

49. The method of claim 46, wherein the region of interest of a biological tissue is a skin lesion.

50. The method of claim 46, wherein the region of interest of a biological tissue is a wound.

51. The method of claim 46, wherein the region of interest of a biological tissue is a burn.

52. The method of claim 46 wherein the digital image of the biological tissue is an image acquired with light in the ultraviolet to blue spectral band.

53. The method of claim 52 wherein:

the region of interest of a biological tissue is a skin lesion; and

prior to the computing step, the digital image in any spectral band is first segmented to locate the region of interest in the image, and

wherein said computing step further comprises calculating at least one quantity chosen from the group consisting of;

the average of the area of dermal papillae within the segmentation mask;

the standard deviation of the area of dermal papillae within the segmentation mask;

average of the thickness of rete ridges within the segmentation mask; and

the standard deviation of the thickness of rete ridges within the segmentation mask.

54. The method of claim 52 wherein:

the region of interest of a biological tissue is a skin lesion; and

prior to the computing step, the digital image in any spectral band is first segmented by means of a segmentation mask, said mask being generated from the digital image acquired in that spectral band for which the imaged skin lesion has the largest area.

55. The method of claim 46 wherein an image is acquired by a digital camera in a least one infrared spectral band.

56. The method of claim 46 wherein the images are acquired in different spectral bands in the visible region by digitizing color slides of the region of interest of the tissue.

57. The method of claim 46, wherein the computing step comprises calculating at least one quantity which is a statistical measure of texture in a portion of the region of interest, wherein the measure is chosen from the group consisting of:

the number of wavelet maxima per unit area;

the ratio of the mean magnitude of the wavelet maxima coefficients to the absolute deviation of the coefficient magnitudes with respect to the mean magnitude;

the ratio of the mean magnitude of the wavelet maxima coefficients to the standard deviation of the coefficient magnitudes; and

the skewness of the coefficient magnitudes, normalized to the cube of the standard deviation;

wherein said portion of the region of interest comprises either of a border subregion and a subregion interior to said border subregion.

58. The method of claim 57, further comprising calculating either of the degree of change of a statistic of the wavelet maxima coefficient distribution upon an increment of wavelet level, and the degree of deviation of such change from linearity.

59. The method of claim 57, further comprising calculating the average rate of change, with respect to level, of the number of wavelet maxima per unit area.

60. The method of claim 46, further comprising comparing each of the computed quantities to a threshold, which threshold is predetermined through statistical analysis of multiscale wavelet maxima representations of a training set of digital images of similar biological tissue, for which the condition of the tissue is independently known.

61. The method of claim 46, wherein prior to the computing step, the digital image is first segmented to locate the region of interest in the image, and said computing step further comprises:

determining the principal axis of the segmented image;

rotating the principal axes of the segmented image until they are oriented parallel to the coordinate axes of the image;

computing the differences in intensity between each pair of pixels whose locations, with respect to a principal axis, are mirror images of each other;

summing the absolute values of said intensity differences;

calculating asymmetry values with respect to each principal axis, by normalizing the said sum to the total intensity in the segmented images; and

adding together the asymmetry values calculated for the two principal axes.

62. The method of claim 61, wherein prior to the "determining" step, the intensities in the digital image are replaced by the binary values of the segmentation mask.

63. The method of claim 46, wherein prior to the computing step, the digital image is first segmented to locate the region of interest in the image, and said computing step further comprises:

calculating at least one quantity chosen from the group consisting of;

a statistical measure of the deviation of the border of the region of interest from the border of an ellipse having the same area, aspect ratio, and orientation as the segmentation mask; and

a statistical measure of the gradients of the intensity values of the digital image across the border of the segmented image.

64. The method of claim 46 wherein the characterizing step comprises the logical combination of the outputs of several intermediate distinguishing steps, wherein each distinguishing step distinguishes between a particular condition of a region of interest of the biological tissue and a different one of several alternative conditions of the same region of interest.

65. The method of claim 46, wherein the digital images are formed by digitizing a color slide of the region of interest in different spectral bands.

66. The method of claim 46, wherein the region of interest of a biological tissue is a pigmented region suspected of melanoma.