Systems and methods for detection of blank fields in digital microscopes

• US 10,444,486 B2
• Filed: 09/04/2017
• Issued: 10/15/2019
• Est. Priority Date: 09/04/2017
• Status: Active Grant

• Current Assignee: Microscopes International LLC
• Original Assignee: Microscopes International LLC
• Inventors: Rainbolt, Richard K., Ward, Jonathan C.
• Primary Examiner(s): Hance, Robert J
• Application Number:
  US15/694,990
• Publication Number:
  US 20190072751A1
• Time in Patent Office: 771 Days
• CPC Class Codes: G01Q 20/00 Monitoring the movement or position of the probe
• CPC Class Codes: G02B 21/002 Scanning microscopes scanning near field optical microscopes G01Q60/18
• CPC Class Codes: G02B 21/26 Stages; Adjusting means therefor
• CPC Class Codes: G02B 21/365 Control or image processing arrangements for digital or video microscopes G02B21/361, G02B21/362 take precedence
• CPC Class Codes: G02B 21/367 providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 2207/10061 from scanning electron microscope
• CPC Class Codes: G06T 2207/20072 Graph-based image processing
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 2207/30096 Tumor; Lesion
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 23/12 with one sensor only
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

Systems and methods for rapidly identifying blank fields while capturing a plurality of color field images using a slide scanning microscope having a movable slide stage, a movable objective lens, and a digital video camera having a color digital image sensor that encodes RGB color data for each pixel in a field image.

Claims

1. A method of identifying blank fields while capturing a plurality of color field images using a slide scanning microscope, each field image comprising an image of a portion of a target area of a microscope slide specimen area, the plurality of field images covering the target area, the slide scanning microscope having a moveable slide stage for holding a microscope slide, a movable objective lens, a digital video camera having a color digital image sensor that encodes RGB color data for each pixel in a field image, and a control unit for controlling the operation of the movable slide stage, the movable objective lens, and the digital video camera in capturing field images, the method comprising:

a) performing a slide stage movement to move the slide stage to a desired stationary field position, the slide stage movement having a slide stage settle time following the slide stage movement, during which vibration of the slide stage microscope precludes capturing a usable image of a non-blank field using the digital video camera;

b) capturing a first color field image at the desired stationary field position using the digital video camera;

c) determining a grayscale pixel value for each pixel in the first color field image based on the RGB color data for the first color field image;

d) determining a first extreme grayscale pixel value as the first grayscale pixel value exceeding a first extreme grayscale threshold number of pixels;

e) determining a second extreme grayscale pixel value as the last grayscale pixel value exceeding a second extreme grayscale threshold number of pixels;

f) determining a first grayscale pixel range distance between the first extreme grayscale pixel value and the second extreme grayscale pixel value; and

g) initiating a repeat of step a) prior to the lapse of the slide stage settle time if the first grayscale pixel range distance is less than a non-blank grayscale pixel range threshold distance.

2. The method of claim 1, further comprising:

h) if the first grayscale pixel range distance is not less than the non-blank grayscale pixel range threshold distance,

1) waiting until the lapse of the slide stage settle time for the first color field image;

2) capturing a second color field image for the desired stationary field position after the lapse of the slide stage settle time for the first color field image, using the digital video camera;

3) performing an objective lens movement to a desired stationary objective lens position for the desired stationary field position;

4) capturing a color field image at the desired stationary objective lens position after the objective lens movement using the digital video camera;

5) repeating steps 3 and 4 one or more times to obtain, together with the second color field image, a Z-stack of color field images at the desired stationary field position, each color field image in the Z-stack of field images comprising an image having a different focus from the other images in the Z-stack; and

6) retaining one color field image of the Z-stack of field images as the color field image for the desired stationary field position.

3. The method of claim 2, further comprising:

i) repeating steps a) to h) to capture each of the plurality of color field images; and

j) assembling the plurality of color field images to form a single color image of the target area.

4. The method of claim 2, wherein retaining one color field image of the Z-stack of color field images comprises

determining a focus value for each color field image in the Z-stack;

and

retaining one color field image of the Z-stack based on the focus value of each of the color field images in the Z-stack.

5. The method of claim 2, wherein the objective lens movement comprises an objective lens settle time, and capturing a color field image after the objective lens movement comprises capturing a color field image after the objective lens settle time has elapsed.

6. The method of claim 1, further comprising, after determining a grayscale pixel value for each pixel in the first color field image based on the RGB color data;

determining the number of pixels of each grayscale pixel value in the first color field image.

7. The method of claim 6, further comprising, after determining a grayscale pixel value for each pixel in the first color field image based on the RGB color data;

preparing a histogram of grayscale pixel values, wherein the histogram includes the number of pixels of each grayscale pixel value in the first color field image.

8. The method of claim 6, wherein determining a first extreme grayscale pixel value comprises determining the first grayscale pixel value having a number of pixels exceeding a first extreme grayscale threshold number of pixels.

9. The method of claim 1, further comprising identifying the stationary field position as a blank field if the first grayscale pixel range distance is less than the non-blank grayscale pixel range threshold distance.

10. The method of claim 1, further comprising:

h) if the first grayscale pixel range distance is not less than the non-blank grayscale pixel range threshold distance;

1) waiting until the lapse of the slide stage settle time; and

2) capturing a second color field image for the desired stationary field position after the lapse of the slide stage settle time using the digital video camera; and

3) retaining the second color field image as the color field image for the desired stationary field position.

11. The method of claim 10, further comprising:

g) repeating steps a) to h) to capture each of the plurality of color field images; and

h) assembling the plurality of color field images to form a single color image of the target area.

12. The method of claim 1 wherein the slide stage movement comprises a settle time of from 30 mSec to 300 mSec, and wherein initiating a repeat of step a) comprises initiating a repeat of step a) within a range of 30 mSec to 300 mSec of the completion of the slide stage movement if the first grayscale pixel range distance is less than a non-blank grayscale pixel range threshold distance.

13. The method of claim 1 wherein the slide stage movement comprises at least one of an X-axis movement having an X-axis settle time of from 100 mSec to 200 mSec and a Y-axis movement having a settle time of from 30 mSec to 100 mSec, and wherein initiating a repeat of step a) comprises initiating a repeat of step a) within the X-axis settle time of an X-axis movement and initiating a repeat of step a) within the Y-axis settle time of a Y-axis movement, if the first grayscale pixel range distance is less than a non-blank grayscale pixel range threshold distance.

14. The method of claim 1, further comprising the step of preparing a grayscale histogram based on the grayscale pixel values determined in step c), and wherein steps d), e), and f) are determined from the grayscale histogram.

15. A method of identifying blank fields while capturing a plurality of color field images using a slide scanning microscope, each field image comprising an image of a portion of a target area of a microscope slide specimen area, the plurality of field images covering the target area, the slide scanning microscope having a moveable slide stage for holding a microscope slide, a movable objective lens, a digital video camera having a color digital image sensor that encodes RGB color data for each pixel in a field image, and a control unit for controlling the operation of the movable slide stage, the movable objective lens, and the digital video camera in capturing field images, the method comprising:

a) performing a slide stage movement to move the slide stage to a desired stationary field position, the slide stage movement having a slide stage settle time following the slide stage movement, during which vibration of the slide stage microscope precludes capturing a usable image of a non-blank field using the digital video camera;

b) capturing a first color field image at the desired stationary field position using the digital video camera;

c) determining a grayscale pixel value for each pixel in the first color field image based on the RGB color data for the first color field image;

d) performing a grayscale pixel value distribution analysis of the first color field image, based on the grayscale pixel values determined for the pixels in the first color field image;

e) determining whether the first color field image is a blank field based on the grayscale pixel value distribution analysis; and

f) initiating a repeat of step a) prior to the lapse of the slide stage settle time, in response to a determination that the first field image is a blank field.

16. The method of claim 15, further comprising:

g) in response to a determination that the first color field image is not a blank field;

1) waiting until the lapse of the slide stage settle time for the first color field image;

2) capturing a second color field image for the desired stationary field position after the lapse of the slide stage settle time for the first color field image, using the digital video camera;

3) performing an objective lens movement to a desired stationary objective lens position for the desired stationary field position;

4) capturing a color field image at the desired stationary objective lens position after the objective lens movement using the digital video camera;

5) repeating steps 3 and 4 one or more times to obtain, together with the second color field image, a Z-stack of color field images at the desired stationary field position, each color field image in the Z-stack of color field images comprising an image having a different focus from the other images in the Z-stack; and

6) retaining one color field image of the Z-stack of color field images as the color field image for the desired stationary field position.

17. The method of claim 16, further comprising:

h) repeating steps a) to g) to capture each of the plurality of color field images; and

i) assembling the plurality of color field images to form a single color image of the target area.

18. The method of claim 15, further comprising:

g) in response to a determination that the first color field image is not a blank field;

1) waiting until the lapse of the slide stage settle time;

2) capturing a second color field image for the desired stationary field position after the lapse of the slide stage settle time using the digital video camera; and

3) retaining the second color field image as the color field image for the desired stationary field position.

19. The method of claim 18, further comprising:

h) repeating steps a) to g) to capture each of the plurality of color field images; and

i) assembling the plurality of color field images to form a single color image of the target area.

20. The method of claim 15, wherein performing a grayscale pixel value distribution analysis comprisesdetermining a first extreme grayscale pixel value as the first grayscale pixel value exceeding a first extreme grayscale threshold number of pixels;

2) determining a second extreme grayscale pixel value as the last grayscale pixel value exceeding a second extreme grayscale threshold number of pixels;

3) determining a first grayscale pixel range distance between the first extreme grayscale pixel value and the second extreme grayscale pixel value; and

4) comparing the first grayscale pixel range distance to a non-blank grayscale pixel range threshold distance.

21. The method of claim 20, wherein determining a first extreme grayscale pixel value comprises determining the first grayscale pixel value having a number of pixels exceeding a first extreme grayscale threshold number of pixels, and wherein determining a second extreme grayscale pixel value comprises determining the last grayscale pixel value having a number of pixels exceeding a second extreme grayscale threshold number of pixels.

22. A method of identifying blank field images in the capture of a plurality of color field images using a slide scanning microscope, each field image comprising an image of a portion of a target area of a microscope slide specimen area, the plurality of field images covering the target area, the slide scanning microscope having a moveable slide stage for holding a microscope slide, at least one objective lens, at least one digital video camera having a color digital image sensor that encodes RGB color data for each pixel in a field image, and a control unit for controlling the operation of the movable slide stage, the at least one objective lens, and the at least one digital video camera in capturing field images, the method comprising:

a) performing a slide stage movement to move the slide stage to a desired stationary field position, the slide stage movement having a slide stage settle time following the slide stage movement, during which vibration of the slide stage microscope precludes capturing a usable image of a non-blank field using the digital video camera;

b) capturing a first color field image at the desired stationary field position using the digital video camera;

c) performing a grayscale pixel value distribution analysis of the first field image comprising

1) determining a grayscale pixel value for each pixel in the first color field image based on the RGB color data for the first color field image;

2) determining the number of pixels of each grayscale pixel value in the first color field image;

3) determining a first extreme grayscale pixel value as the first grayscale pixel value having a number of pixels exceeding a first extreme grayscale threshold number of pixels;

4) determining a second extreme grayscale pixel value as the last grayscale pixel value having a number of pixels exceeding a second extreme grayscale threshold number of pixels;

5) determining a first grayscale pixel range distance as the distance between the first extreme grayscale pixel value and the second extreme grayscale pixel value;

6) comparing the first grayscale pixel range distance to a non-blank grayscale pixel range threshold distance; and

d) initiating a repeat of step a) prior to the lapse of the slide stage settle time if the first grayscale pixel range distance is less than the non-blank grayscale pixel range threshold distance.

23. The method of claim 22 wherein the at least one objective lens of the slide scanning microscope comprises a movable objective lens, the method further comprising:

e) in response to a determination that the first grayscale pixel range distance is not less than the non-blank grayscale pixel range threshold distance;

1) waiting until the lapse of the slide stage settle time;

2) capturing a second color field image for the desired stationary field position after the lapse of the slide stage settle time for the first color field image, using the digital video camera;

3) performing an objective lens movement to a desired stationary objective lens position for the desired stationary field position;

4) capturing a color field image at the desired stationary objective lens position after the objective lens movement using the digital video camera;

5) repeating steps 3 and 4 one or more times to obtain, together with the second color field image, a Z-stack of color field images at the desired stationary field position, each color field image in the Z-stack of field images comprising an image having a different focus from the other images in the Z-stack; and

6) retaining one color field image of the Z-stack of color field images as the color field image for the desired stationary field position.

24. The method of claim 23, further comprising:

f) repeating steps a) to e) to capture each of the plurality of color field images; and

g) assembling the plurality of color field images to form a single color image of the target area.

25. The method of claim 22, further comprising:

e) in response to a determination that the first grayscale pixel range distance is not less than the non-blank grayscale pixel range threshold distance;

1) waiting until the lapse of the slide stage settle time;

2) capturing a second color field image for the desired stationary field position after the lapse of the slide stage settle time using the digital video camera; and

3) retaining the second color field image as the color field image for the desired stationary field position.

26. The method of claim 22, further comprising:

determining that the first color field image is a blank field if the first grayscale pixel range distance is less than the non-blank grayscale pixel range threshold distance; and

determining that the first color field image is not a blank field if the first grayscale pixel range distance is not less than the non-blank grayscale pixel range threshold distance.

27. The method of claim 22, further comprising, after determining a grayscale pixel value for each pixel in the first color field image based on the RGB color data:

preparing a histogram of grayscale pixel values, wherein the histogram includes the number of pixels of each grayscale pixel value in the first color field image.

28. The method of claim 22, further comprising retaining the first color field image as the field image for the desired stationary field position if the first grayscale pixel range distance is less than the non-blank grayscale pixel range threshold distance.