Systems and methods for the multispectral imaging and characterization of skin tissue

• US 6,208,749 B1
• Filed: 02/27/1998
• Issued: 03/27/2001
• Est. Priority Date: 02/28/1997
• Status: Expired due to Term

• Current Assignee: Mela Sciences, Inc.
• Original Assignee: Electro-Optical Sciences, Inc.
• Inventors: Gutkowicz-Krusin, Dina, Jacobs, Adam, Elbaum, Marek, Greenebaum, Michael
• Primary Examiner(s): Au, Amelia
• Assistant Examiner(s): Dastouri, Mehrdad
• Application Number:
  US09/032,450
• Time in Patent Office: 1,124 Days
• Field of Search: 382/128, 382/162, 382/173, 382/274, 382/165, 358/504, 356/303, 356/346
• US Class Current: 382/128
• CPC Class Codes: A61B 2576/02 specially adapted for a particular organ or body part
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/442 Evaluating skin mechanical properties, e.g. elasticity, hardness, texture, wrinkle assessment
• CPC Class Codes: A61B 5/445 Evaluating skin irritation or skin trauma, e.g. rash, eczema, wound, bed sore
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing

Abstract

Systems and methods for the multispectral imaging of skin tissue enables automatic characterization of the condition of a region of interest of the skin, based on direct digital imaging of the region of interest or the digitization of color photographic slides of the region of interest, illuminated by appropriately filtered light. Preferably, a digital image at a low spectral band is automatically segmented and that segmented mask is used to segment the other images by a digital processor. Parameters related to the texture, asymmetry, blotchiness and border irregularities are also automatically estimated. The region of interest is automatically characterized by the digital processor, based on those parameters. The region of interest may include a skin lesion, in which case the present invention enables the characterization of the lesion as malignant or benign.

Claims

1. A method of characterizing a skin lesion wherein the absorption and scab of light in different spectral bands by the skin lesion is a function of the condition of the skin, the method comprising:

illuminating a portion of the skin including the region of interest by light in at least three spectral bands;

digitally imaging a portion of the skin including the region of interest at the at least three spectral bands with the light re-emitted by the portion of the skin to generate digital images comprising digital signals whose values are a function of the condition of the region of interest of the skin; and

providing the digital images to a processor, wherein the processor;

segments the digital image by generating a single segmentation mask defining the boundary of the region of interest for each image, where the single segmentation mask is the segmentation mask having largest area of segmentation masks generated from each image in each of the at least three spectral bands, without operator intervention;

automatically computes at least one estimated value for each digital image at each spectral band which is a function of a characteristic of the portion of the region of interest determined by the segmentation mask, without operator intervention;

characterizes the condition of the skin as malignant or benign based on the estimated values, without operator intervention; and

outputs the characterization of the condition of the skin.

2. The method of claim 1, further comprising estimating at least one value which is a function of the texture of the region of interest.

3. The method of claim 2, wherein the computing step comprises estimating values which are statistical measures of local intensity variation in the digital images in each spectral band, which are a function of the texture of the region of interest.

4. The method of claim 2, wherein the computing step comprises estimating values based on the ratio of standard deviation of the areas of dermal papillae to their mean within the segmentation mask.

5. The method of claim 2, wherein the computing step comprises estimating values of the average and standard deviation of the thickness of rete ridges within the segmentation masks.

6. The method of claim 1, further comprising estimating at least one value which is a function of the asymmetry of the region of interest in each spectral band, for two principal axes of the segmented image by:

determining the principal axes of the segmented image;

rotating the principal axes of the segmented image until they are oriented parallel to the coordinate axes of the image;

computing the differences in intensity between each pair of pixels whose locations, with respect to a principal axis, are mirror images of each other;

summing the absolute values of said intensity differences;

calculating asymmetry values with respect to each principal axis, by normalizing the said sum to the total intensity in the segmented images; and

adding together the asymmetry values calculated for the two principal axes.

7. The method of claim 1, further comprising estimating at least one value which is a function of the blotchiness of the region of interest.

8. The method of claim 1, further comprising estimating at least one value which is a function of the irregularity of the border of the region of interest by estimating a value which is a statistical measure of the deviation of the border of the segmentation mask from the border of an ellipse of the same area, aspect ratio, and orientation as the segmentation mask.

9. The method of claim 1, further comprising estimating a value which is a function of the gradient at the border of the region of interest by estimating a statistical measure of the gradient values of the intensity of the digital images across the border of the segmented images, at each spectral band.

10. The method of claim 1, further comprising characterizing the type of lesion as invasive or non-invasive.

11. The method of claim 1, wherein the segmenting step comprises generating the segmentation mask from a digital image by:

removing digital signals from the digital image which correspond to hair structures;

deriving a threshold from a multimodal histogram of intensity levels;

iteratively applying the threshold to the digital signals of the digital image; and

removing digital signals which correspond to small blob-like regions from the masked image.

12. The method of claim 1, wherein the digital imaging step further comprises digitally imaging the region of interest with a digital camera.

13. The method of claim 1, further comprising:

photographing the region of interest with a color camera to form color photographic slides; and

illuminating the color photographic slides with light in each spectral band;

wherein the digital imaging step comprises digitally imaging the illuminated color photographic slides of the region of interest with a digital camera.

14. The method of claim 1, where the characterization step is based on a non-linear combination of the estimated values.

15. The method of claim 1, where the characterization step is based on a linear combination of the estimated values.

16. The method of claim 1, where the characterization step is based on a sequential combination of applying a linear combination of the estimated values and a non-linear combination of estimated values.

17. A method of characterizing the condition of a region of interest of skin, wherein the absorption and scattering of light in different spectral bands by the region of interest is a function of the condition of the skin, the method comprising:

illuminating a portion of the skin including the region of interest by light in at least thee spectral bands;

digitally imaging the portion of the skin including the region of interest at the at least tree spectral bands with the light re-emitted by the portion of the skin to generate digital images comprising digital signals whose values are a function of the condition of the region of interest of the skin; and

providing the digital images to a processor, wherein the processor;

segments the digital images by generating a single segmentation mask defining the boundary of the region of interest for each image, where the single segmentation mask is the segmentation mask having largest area of segmentation masks generated from each image in each of the at least three spectral bands;

computes at least one estimated value for each digital image at each spectral band which is a function of a characteristic of the region of interest determined by the segmentation mask;

characterizes the condition of the region of interest of the skin based on the estimated values; and

outputs the characterization of the condition of the region of interest of the skin.

18. The method of claim 17, wherein the estimating and characterizing steps are conducted without the intervention of an operator.

19. The method of claim 17, wherein the segmenting step is conducted without the intervention of an operator.

20. The method of claim 17, wherein the illuminating step further comprises illuminating the region of interest with light in at least one spectral band which penetrates to the papillary dermis and is re-emitted therefrom.

21. The method of claim 20, wherein the digital imaging step further comprises digitally imaging the region of interest with a digital camera.

22. The method of claim 20, wherein the illuminating step further comprises illuminating the region of interest with light in a near infrared spectral band.

23. The method of claim 17, further comprising suppressing specular reflections prior to the digital imaging step.

24. The method of claim 23, wherein the processor converts the digital signals of each of the digital images into values corrected for non-uniformities of illumination and of response prior to the segmenting step.

25. The method of claim 17, further comprising:

photographing the region of interest with a color camera to form color photographic slides; and

illuminating the color photographic slides with light in each spectral band;

wherein the digital imaging step comprises digitally imaging the illuminated color photographic slides of the region of interest with a digital camera.

26. The method of claim 17, wherein the segmenting step further comprises applying the segmentation mask to the digital images in the other spectral bands.

27. The method of claim 17, wherein the segmenting step comprises generating the mask at the shortest available wavelength.

28. The method of claim 17, wherein the illuminating step comprises illuminating the region of interest by light in at least one spectral band whose center is between about 400 to about 500 nanometers, and the segmenting step comprises generating the mask from a digital image at the spectral band between about 400 to about 500 nanometers.

29. The method of claim 19, wherein the segmenting step comprises generating the segmentation mask from a digital image by:

removing digital signals from the digital image which correspond to hair structures;

deriving a threshold from a multimodal histogram of intensity levels;

iteratively applying the threshold to the digital signals of the digital image; and

removing digital signals which correspond to small blob-like regions from the masked image.

30. The method of claim 19, wherein the segmenting step comprises generating the segmentation mask by comparing estimated values which are a function of textures within the digital images with a threshold.

31. The method of claim 30, further comprising generating the segmentation mask by comparing the estimated texture values to a threshold derived through statistical analysis of each digital image.

32. The method of claim 17, wherein the computing step comprises estimating at least one value which is a function of the texture of the region of interest determined by the segmentation mask.

33. The method of claim 32, wherein the computing step further comprises estimating values which are a function of the texture of the region of interest determined by the segmentation mask separately in each spectral band, based on the same segmentation mask.

34. The method of claim 32, wherein the computing step comprises estimating values which are statistical measures of local intensity variation in the digital images in each spectral band which are a function of texture.

35. The method of claim 17, wherein the computing step further comprises estimating a value which is a function of the asymmetry of the segmented image in each spectral band, for two principal axes of the segmented image.

36. The method of claim 35, wherein the computing step further comprises:

locating the principal axes by computing an orientation angle;

computing the intensity centroid;

rotating the digital image such that the principal axes are parallel to the image axes;

estimating asymmetry values for each principal axis based on the intensity centroid; and

summing the estimated asymmetry values for the two principal axes.

37. The method of claim 36, wherein the computing step further comprises computing the intensity moment with a binary intensity distribution.

38. The method of claim 17, wherein the computing step further comprises estimating at least one value which is a function of the blotchiness of the segmented digital image, the estimated blotchiness value being defined through statistical properties of the spatial distribution of topographic regions of the digital images at each spectral band.

39. The method of claim 38, wherein the computing step further comprises determining the centroids of topographic regions of the segmented digital image at each spectral band.

40. The method of claim 17, wherein the characterizing step comprises comparing a weighted combination of estimated values against a threshold value.

41. The method of claim 40, wherein the condition of the region of interest to be characterized is the presence of a melanoma and weight coefficients for each estimated value and the threshold value are selected to maximize specificity, under the constraint of a defined sensitivity to melanoma, on a representative set of training images.

42. The method of claim 17, where the characterization step is based on a non-linear combination of the estimated values.

43. The method of claim 17, where the characterization step is based on a linear combination of the estimated values.

44. The method of claim 17, where the characterization step is based on a sequential combination of applying a linear combination of the estimated values and a non-linear combination of estimated values.

45. A system for characterizing the condition of a region of interest of skin, comprising:

a source of illumination of light in at least three spectral bands;

a camera for acquiring digital images of the region of interest based on the light re-emitted from the illuminated region of interest at each of the spectral bands, the digital image comprising digital signals whose values are a function of the condition of the region of interest;

memory for storing the digital images provided by the camera;

a digital processor programmed to perform the steps of;

segmenting the digital images stored in memory by generating a single segmentation mask, where the single segmentation mask is the segmentation mask having largest area of segmentation masks generated from each image in each of the at least three spectral bands;

estimating at least one value for each digital image at each spectral band which is a function of the texture of the portion of the region of interest determined by the segmentation mask;

characterizing the condition of the skin based on the estimated values; and

outputting the characterization of the region of interest.

46. The system of claim 45, further comprising means for suppressing specular reflections from the region of interest.

47. The system of claim 45, further comprising means for calibrating each digital image to provide correction for non-uniformities of illumination and response.

48. The system of claim 45, wherein the digital processor is coupled to the source of illumination and to the camera for controlling the intensity of illumination and exposure times, respectively.

49. The system of claim 45, wherein the processor applies the segmentation mask derived from the digital images at one spectral band to the digital images at the other spectral bands.

50. The system of the claim 45, where in the processor estimates values separately from digital images at each spectral band based on the segmentation mask.

51. The system of claim 49, wherein the processor compares a weighted combination of estimated values against a threshold value.

52. The system of claim 45, wherein the camera records monochromatic images and the illumination means comprises:

a tungsten halogen light source with feedback to stabilize the intensity in each wavelength band;

means for sequentially filtering the light; and

an optical fiber ring illuminator to distribute the filtered light.

53. The system of claim 45, further comprising a feedback loop for stabilizing the intensity of the light source by the processor.

54. The system of claim 45, wherein the filter means comprises a plurality of interference filters mounted on a wheel for stepping any filter into a position intercepting the light from the light source.

55. The system of claim 45, wherein at least one of the spectral bands has a center which is between about 400 to about 500 nanometers, and at least one other band centered elsewhere in the visible region.

56. The system of claim 55, wherein the set of interference filters includes a filter whose center lies in at least one spectral band in the near infrared range whose center lies between about 750 and 1000 nanometers.

57. The system of claim 50, wherein the camera is a single-chip, charge-coupled device and the control means comprises a digital computer including means for determining exposure times for the camera which maximize the signal-to-noise ratio in the image at each spectral band.

58. The system of claim 45, wherein:

the source of illumination provides broad-band (“

white”

) light; and

the camera comprises multiple charge-coupled devices which detect light in a plurality of spectral bands between the near ultraviolet to near infrared.

59. The system of claim 45, wherein the processor estimates values which are statistical measures of local intensity variation in the digital images in each spectral band, which are a function of the texture of the region of interest.

60. The system of claim 45, wherein the processor estimates values based on the ratio of standard deviation of the areas of dermal papillae to their mean within the segmentation mask.

61. The system of claim 45, wherein the processor estimates values of the average and standard deviation of the thickness of rete ridges within the segmentation masks.

62. The system of claim 45, wherein the processor estimates at least one value which is a function of the asymmetry of the region of interest in each spectral band, for two principal axes of the segmented image by:

locating the principal axes by computing an orientation angle;

computing the intensity centroid;

rotating the digital image such that the principal axes are parallel to the image axes; and

estimating asymmetry values for each principal axis based on the intensity centroid; and

summing the estimated asymmetry values for the two principal axes.

63. The system of claim 45, wherein the processor further estimates at least one value which is a function of the blotchiness of the region of interest.

64. The system of claim 45, wherein the processor further estimates at least one value which is a function of the irregularity of the border of the region of interest by estimating a value which is a statistical measure of the deviation of the border of the segmentation mask from the border of an ellipse of the same area, aspect ratio, and orientation as the segmentation mask.

65. The system of claim 45, wherein the processor further estimates a value which is a function of the gradient at the border of the region of interest by estimating a statistical measure of the gradient values of the intensity of the digital images across the border of the segmented images, at each spectral band.

66. The system of claim 45, wherein the processor characterizes the type of lesion as invasive or non-invasive.

67. The system of claim 45, wherein the processor generates the segmentation mask from a digital image by:

removing digital signals from the digital image which correspond to hair structures;

deriving a threshold from a multimodal histogram of intensity levels;

iteratively applying the threshold to the digital signals of the digital image; and

removing digital signals which correspond to small blob-like regions from the masked image.

68. A system for characterizing the condition of a region of interest of skin, comprising:

a source of illumination of light in at least three spectral bands;

a camera for acquiring digital images of the region of interest based on the light re-emitted from the illuminated region of interest at each of the at least three spectral bands, the digital image comprising digital signals whose values are a function of the condition of the region of interest;

a memory for storing the digital images;

a digital processor including;

digital processing means for segmenting the digital images stored in memory and computing estimated values of parameters which are a function of the segmented images, wherein the digital images are segmented by generating a single segmentation mask, where the single segmentation mask is the segmentation mask having largest area of segmentation masks generated from each image in each of the at least three spectral bands;

digital processing means for automatically characterizing the condition of the tissue based on the estimated values; and

means for outputting the characterization of the region of interest.

69. A method of characterizing the condition of a region of interest of skin, wherein the absorption and scattering of light in different spectral bands by the region of interest is a function of the condition of the skin, the method comprising:

illuminating a portion of the skin including the region of interest by light in at least three spectral bands;

digitally imaging the portion of the skin including the region of interest at the at least three spectral bands with the light re-emitted by the portion of the skin to generate digital images comprising digital signals whose values are a function of the condition of the region of interest of the skin; and

providing the digital images to a processor, wherein the processor;

segments the digital images by generating a segmentation mask defining the boundary of the region of interest from a digital image in any one of the at least three spectral bands;

computes at least one estimated value which is a statistical measure of the deviation of the boundary of the region of interest from the boundary of an ellipse of the same area, aspect ratio, and orientation as the segmentation mask;

characterizes the condition of the region of interest of the skin based on the estimated values; and

outputs the characterization of the condition of the region of interest of the skin.

70. A method of characterizing the condition of a region of interest of skin, wherein the absorption and scattering of light in different spectral bands by the region of interest is a function of the condition of the skin, the method comprising:

illuminating a portion of the skin including the region of interest by light in at least three spectral bands;

digitally imaging the portion of the skin including the region of interest at the at least three spectral bands with the light re-emitted by the portion of the skin to generate digital images comprising digital signals whose values are a function of the condition of the region of interest of the skin; and

providing the digital images to a processor, wherein the processor;

segments the digital images by generating a segmentation mask defining the boundary of the region of interest from a digital image in any one of the at least three spectral bands;

computes at least one estimated value of a statistical measure of the gradient values of the intensity of the digital images across the border of the segmented images;

characterizes the condition of the region of interest of the skin based on the estimated values; and

outputs the characterization of the condition of the region of interest of the skin.

71. A method of characterizing the condition of a region of interest of skin, wherein the absorption and scattering of light in different spectral bands by the region of interest is a function of the condition of the skin, the method comprising:

illuminating a portion of the skin including the region of interest by light in at least three spectral bands;

digitally imaging the portion of the skin including the region of interest at the at least three spectral bands with the light re-emitted by the portion of the skin to generate digital images comprising digital signals whose values are a function of the condition of the region of interest of the skin; and

providing the digital images to a processor, wherein the processor;

segments the digital images by generating a segmentation mask defining the boundary of the region of interest from a digital image in any one of the at least three spectral bands;

computes at least one estimated value based on the ratio of standard deviation of the areas of dermal papillae to their mean within the segmentation mask;

characterizes the condition of the region of interest of the skin based on the estimated values; and

outputs the characterization of the condition of the region of interest of the skin.

72. A method of characterizing the condition of a region of interest of skin, wherein the absorption and scattering of light in different spectral bands by the region of interest is a function of the condition of the skin, the method comprising:

illuminating a portion of the skin including the region of interest by light in at least three spectral bands;

digitally imaging the portion of the skin including the region of interest at the at least three spectral bands with the light re-emitted by the portion of the skin to generate digital images comprising digital signals whose values are a function of the condition of the region of interest of the skin; and

providing the digital images to a processor, wherein the processor;

segments the digital images by generating a segmentation mask defining the boundary of the region of interest from a digital image in any one of the at least three spectral bands;

computes at least one estimated value of the average and standard deviation of the thickness of rete ridges within the segmentation mask for a digital image of the region of interest determined by the segmentation mask;

characterizes the condition of the region of interest of the skin based on the estimated values; and

outputs the characterization of the condition of the region of interest of the skin for a digital image of the region of interest determined by the segmentation mask.

73. A method of characterizing the condition of a region of interest of skin, wherein the absorption and scattering of light in different spectral bands by the region of interest is a function of the condition of the skin, the method comprising:

illuminating a portion of the skin including the region of interest by light in at least three spectral bands;

digitally imaging the portion of the skin including the region of interest at the at least three spectral bands with the light re-emitted by the portion of the skin to generate digital images comprising digital signals whose values are a function of the condition of the region of interest of the skin;

calibrating each pixel location in the digital image in each spectral band with respect to stored images of a white target material having known diffuse reflectance, each of the stored images being an average of a plurality of images acquired at each spectral band, while the material undergoes continual in-plane motion; and

providing the digital images to a processor, wherein the processor;

segments the digital images by generating a segmentation mask defining the boundary of the region of interest from a digital image in any one of the at least three spectral bands;

computes at least one estimated value for each digital image at each spectral band which is a function of a characteristic of the region of interest determined by the segmentation mask;

characterizes the condition of the region of interest of the skin based on the estimated values; and

outputs the characterization of the condition of the region of interest of the skin.