Method for identifying objects using data processing techniques

• US 5,528,703 A
• Filed: 01/10/1994
• Issued: 06/18/1996
• Est. Priority Date: 02/18/1992
• Status: Expired due to Term

• Current Assignee: Tripath Imaging Incorporated (Becton, Dickinson & Co)
• Original Assignee: NeoPath, Inc. (Becton, Dickinson & Co)
• Inventors: Lee, Shih-Jong J.
• Primary Examiner(s): Pascal, Leslie
• Assistant Examiner(s): Do, Anh Hong
• Application Number:
  US08/179,812
• Time in Patent Office: 890 Days
• Field of Search: 382/6, 382/1, 382/9, 382/55, 382/54, 382/8, 382/22, 382/283, 382/257, 382/155
• US Class Current: 382/257
• CPC Class Codes: G01N 15/1433 using image recognition
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/155 involving morphological operators
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 20/69 Microscopic objects, e.g. biological cells or cellular parts

Abstract

The present invention provides a method for identifying the size, shape, and location of objects in a specimen wherein the image of the specimen is represented by image data and wherein the image data is processed to provide mask data representing a mask wherein the mask identifies the size, shape, and location of the object. Generally, the method includes the step of enhancing the image and creating a threshold image wherein the threshold image includes a threshold intensity value associated with each pixel of the image. The threshold image is combined with the original image to provide a mask image that identifies the size, shape, and location of the objects. The mask image may be further refined to ensure accurate identification of the object. Various other techniques are disclosed within the general method for processing image data.

Claims

1. A method for creating a mask to identify objects of interest wherein the objects of interest are contained in an image represented by an array of image data, said method comprising the steps of:

(a) removing objects from the image that are larger than a predetermined size and performing a histogram to provide parameter data, wherein the parameter data comprises intensity ranges of background range, cytoplasm range and nucleus range, wherein the objects include a texture, and objects of interest have an intensity value within the nucleus range;

(b) enhancing the image contrast by removing the texture of the objects of interest using morphological opening and conditional dilation on the image data to provide data representing a non-textured image wherein the texture of objects of interest is removed only from the image areas having the intensity value within the nucleus range;

(c) normalizing the background of the image represented by the image data to decrease the difference in contrast between the background and the objects of interest to provide data representing a normalized image, wherein image data having an intensity value greater than a background threshold value is set to the background threshold value before normalizing;

(d) combining the normalized image and the non-textured image by filtering the data representing the normalized image in a linear convolution and subtracting the data representing the non-textured image therefrom to provide data representing an enhanced image;

(e) obtaining the difference between the non-textured image and the image to provide a textured image;

(f) processing the data representing the non-textured image to create data representing a threshold image by performing a morphological dilation residue operation on the data representing the textured image and combining the data representing the textured image and the non-textured image with the result to create data representing a threshold image;

(g) creating a low threshold image and a high threshold image by subtracting and adding, respectively, a predetermined offset from the data representing the threshold image;

(h) comparing the data representing the threshold image with the data representing the enhanced image to identify objects of interest by identifying data representing an object of interest as any data representing the enhanced image having a value exceeding the value of the respective data representing the threshold image to create a preliminary mask;

(i) processing the data representing the enhanced image to detect its dark edges, where a variation between a pixel and its neighbor represents the object of interest, and produce data thereof and combining said dark edge data with the preliminary mask to create data representing a dark edge incorporated mask;

(j) processing the data representing the enhanced image to detect its bright edges, where a variation between a pixel and its neighbor represents the pixel outside the object of interest, and combining the resulting data with the dark edge incorporated mask to create data representing a bright edge excluded mask;

(k) filling holes in the bright edge excluded mask by inverting the bright edge excluded mask and excluding data representing an object of size less than the predetermined size by a predetermined amount, the identified objects are then added back to the bright edge excluded mask to create a hole-filled mask;

(l) eroding the hole-filled mask by a first amount, then dilating by a second amount less than the first amount and determining a separation boundary between the identified objects through morphological closing residue operation and then subtracting the separation boundary from the hole-filled mask to create an overlay object separated mask;

(m) comparing the data representing the high threshold image with the enhanced image to create a data representing a high threshold mask, wherein a pixel including the data representing a high threshold mask is included in the high threshold mask if the data'"'"'s respective value in the enhanced image is greater than a respective data value of the high threshold image, and wherein any objects identified by the high threshold mask are added to the object separated mask by a set union operation to create a high threshold included mask;

(n) comparing the data representing the low threshold image with the enhanced image to create a data representing a low threshold mask, wherein a pixel including the data representing a low threshold mask is included in the low threshold mask if the data'"'"'s respective value in the enhanced image is greater than a respective data value of the low threshold image; and

(o) eroding the data representing the high threshold included mask by a small amount and dilating the resulting data by a large amount to connect all objects that are identified by the high threshold included mask and combining the resulting data with the low threshold mask to identify any objects in the low threshold mask not proximate the connected objects of the eroded and dilated high threshold included mask and adding these objects to the original high threshold included mask to create the mask identifying the objects of interest.

2. A method for creating a mask to identify objects of interest wherein the objects of interest are contained in an image represented by an array of image data, said method comprising the steps of:

(a) removing objects from the image data that are larger than a predetermined size, wherein the objects include a texture;

(b) removing the texture of the objects of interest to provide non-textured image data;

(c) normalizing the background of the image data to provide normalized image data;

(d) filtering the normalized image data to generate filtered normalized image data and subtracting the non-textured image data from the filtered normalized image data to provide enhanced image data;

(e) obtaining the difference between the non-textured image data and the image data to provide textured image data;

(f) performing a morphological dilation residue operation on the enhanced image data and combining the textured image data and the non-textured image data with the result to create threshold image data;

(g) creating low threshold image data and high threshold image data by subtracting and adding, respectively, a predetermined offset from the threshold image data;

(h) identifying image data representing an object of interest as any enhanced image data having a value exceeding the value of the respective threshold image data to create a preliminary mask;

(i) detecting the dark edges, where a variation between a pixel and its neighbor represents the object of interest, of the enhanced image data to produce dark edge data and combining the dark edge data with the preliminary mask to create dark edge mask data;

(j) detecting the bright edges, where a variation between a pixel and its neighbor represents the pixel outside the object of interest, of the enhanced image data to produce bright edge data, wherein the bright edge data comprises pixels, and excluding the pixels in the bright edge data from the dark edge mask data to create bright edge mask data;

(k) filling holes in the bright edge mask data by inverting the bright edge mask data and excluding data representing an identified object of size less than the predetermined size by a predetermined amount and adding the identified objects back to the bright edge mask data to create a hole-filled mask;

(l) eroding the hole-filled mask by a first amount and dilating by a second amount less than the first amount and determining a separation boundary between the identified objects through a morphological closing residue operation and subtracting the separation boundary from the hole-filled mask to create an object mask;

(m) comparing the high threshold image data with the enhanced image data, wherein a pixel including the data representing a high threshold mask is included in the high threshold mask if the data'"'"'s respective value in the enhanced image is greater than a respective data value of the high threshold image, and wherein any objects identified by the high threshold mask are added to the object separated mask by a set union operation to create high threshold included mask data;

(n) comparing the low threshold image data with the enhanced image data to create low threshold mask data, wherein a pixel including the data representing a low threshold mask is included in the low threshold mask data if the data'"'"'s respective value in the enhanced image is greater than a respective data value of the low threshold image; and

(o) eroding the high threshold included mask data by a small amount and dilating the resulting data by a large amount to connect all objects that are identified by the high threshold included mask data and combining the resulting data with the low threshold mask data to identify any objects in the low threshold mask data not proximate the connected objects of the eroded and dilated high threshold included mask data and adding these objects to the original high threshold included mask data to create the mask identifying the objects of interest.

3. A method for identifying objects of interest contained in an image represented by image data, wherein the objects of interest have a texture, comprising the steps of:

(a) removing the texture of the objects of interest to provide non-textured image data and textured image data;

(b) normalizing and filtering the background of the image data and subtracting the non-textured image data therefrom to provide enhanced image data;

(c) detecting the edge image of the enhanced image data and combining the textured image data and the non-textured image data with the result to create threshold image data;

(d) creating low threshold image data and high threshold image data by subtracting and adding, respectively, a predetermined offset from the threshold image data;

(e) identifying image data representing an object of interest as any enhanced image data having a value exceeding the value of the respective threshold image data to create a preliminary mask;

(f) detecting the dark edges, where a variation between a pixel and its neighbor represents the object of interests, of the enhanced image data and combining the detected dark edge data with the preliminary mask to create dark edge mask data;

(g) detecting the bright edges, where a variation between a pixel and its neighbor represents the pixel outside the object of interest, of the enhanced image data and excluding pixels in the bright edge data from the dark edge mask to create bright edge mask data;

(h) inverting the bright edge mask data and excluding data representing identified objects of size less than the predetermined size by a predetermined amount and adding back the data representing identified objects to the bright edge mask data to create a hole-filled mask;

(i) determining a separation boundary between the identified objects and subtracting the separation boundary from the hole-filled mask to create an object mask;

(j) comparing the high threshold image data with the enhanced image data to create a data representing a high threshold mask wherein a pixel including the data representing a high threshold mask is included in the high threshold mask if the data'"'"'s respective value in the enhanced image data is greater than a respective data value of the high threshold image data, and wherein any objects identified by the high threshold mask are added to the object separated mask by a set union operation to create a high threshold included mask;

(k) comparing the low threshold image data with the enhanced image data to create low threshold mask data, wherein a pixel including the data representing a low threshold mask is included in the low threshold mask data if the data'"'"'s respective value in the enhanced image is greater than a respective data value of the low threshold image; and

(l) eroding the high threshold mask data by a small amount and dilating the resulting data by a large amount and combining the resulting data with the low threshold mask data and adding these objects to the original high threshold mask data to create the mask identifying the objects of interest.

4. A method for identifying objects of interest contained in an image represented by image data comprising the steps of:

(a) processing the image data to create low threshold image data and high threshold image data wherein the processed image data includes objects identified by the low threshold image data and the high threshold image data;

(b) filling holes in the image data by inverting the image data and excluding data representing identified objects of size less than a predetermined size by a predetermined amount;

(c) detecting the dark edges, where a variation between a pixel and its neighbor represents the object of interest, of the preliminary mask and combining the dark edges, where a variation between a pixel and its neighbor represents the object of interest, with the preliminary mask to create dark edge incorporated mask data; and

(d) detecting the bright edges, where a variation between a pixel and its neighbor represents the pixel outside the object of interest, of the preliminary mask and excluding pixels in the bright edge data from the dark edge incorporated mask data to create data representing a mask identifying the objects of interest.