Method and apparatus for acquiring and reconstructing magnified specimen images from a computer-controlled microscope

• US 20010050999A1
• Filed: 12/28/2000
• Published: 12/13/2001
• Est. Priority Date: 03/03/1997
• Status: Active Grant

• Current Assignee: Evident Scientific Incorporated (Evident Corporation)
• Original Assignee: Bacus Laboratories Incorporated (Olympus Corporation)
• Inventors: Bacus, James W., Bacus, James V.
• Granted Patent: US 6,404,906 B2
• Time in Patent Office: Days
• US Class Current: 382/128
• CPC Class Codes: G01N 15/1433 using image recognition
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06V 20/69 Microscopic objects, e.g. biological cells or cellular parts
• CPC Class Codes: G16H 10/40 for data related to laboratory analysis, e.g. patient specimen analysis
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 40/63 for local operation

Abstract

A method is disclosed for using a computer controlled microscope system to provide a reconstructed, seamless image from several contiguous fields of view to show the architecture and spatial relationship of biological material in a specimen. The specimen is scanned with a microscope and a digital scanner to provide digitized titles of contiguous, fields of view at a predetermined magnification, optical resolution and pixel resolution. Preferably, an automated X, Y stage with higher positional spatial resolution than the spatial pixel resolution of the digital scanner is used to acquire the contiguous fields of view and the image registration information which is used to reconstruct and to display on a monitor the reconstructed image formed of the contiguous multiple fields of view. Preferably, a macro image of contiguous image tiles at a low magnification and optical resolution is obtained and the user navigates within the macro image and selects areas. Higher magnification titles for the selected area are assembled and viewed at a higher magnification and optical resolution as a composite micro image of the selected area. Both the macro and micro images may be displayed simultaneously. Preferably, the user may scroll and cause additional fields of view to be added to previously displayed fields of view.

Claims

1. A method of using a computer-controlled microscope imaging system for acquiring and analyzing areas of interest, said method comprising the steps of:

scanning and digitizing a specimen at a low magnification through a microscope;

displaying to the observer a low magnification, digitized image of the specimen;

selecting a segment of the specimen from the low magnification digitized image for viewing at a higher magnification greater than the low magnification;

automatically scanning the selected segment at the higher magnification and digitizing the image of the segment; and

making available in real time to the view of the observer the respective, low magnification, digitized image and the high magnification, digitized image of the segment.

2. The method of

claim 1

wherein the step of scanning and digitizing at low magnification further includes the steps of:

positioning and coordinating spatially a plurality of low magnification image tiles spatially to provide a composite macro image of the specimen formed from the plurality of low magnification image tiles; and

providing a macro image of the specimens having a field of view substantially larger than the field of view yielded by the objective lens used for the low magnification scanning.

3. A method in accordance with

claim 2

including the step of reducing the size of the composite, macro image before displaying the composite macro image of the specimen to the viewer.

4. A method in accordance with

claim 2

wherein the step of selecting and digitizing the low magnification, digitized image further includes the step of positioning and coordinating spatially a plurality of low magnification images spatially to provide a composite image having a field of view substantially larger than the field of view yielded by the objective lens used for high magnification scanning.

5. A method in accordance with

claim 1

in which the step of the selecting a defined segment area of interest further includes the step of moving a marker on the low magnification image to a point of interest to be viewed at high magnification;

and operating the microscope to change objectives and conditions of illumination to provide a good image at high magnification.

6. A method in accordance with

claim 5

wherein the high and low magnification images are each reconstructed from image tiles each having a spatially known location used to assure their proper positioning in the reconstructed image.

7. A method in accordance with

claim 6

wherein the step of automatically changing from one magnification scanning operation to another magnification scanning operation which includes one or more of the steps of:

changing the microscope objective lens to a different magnification;

changing at least one filter;

changing the field diaphragm;

changing the light intensity;

changing the focus; and

shifting the microscopic stage.

8. The method of

claim 5

including the step of displaying the low magnification image of the specimen and displaying the marker on the low magnification image, and simultaneously displaying the high magnification image of the segment at a position located by the marker on the low magnification image.

9. The method of

claim 1

including the step of shifting the marker to different segments and viewing at high magnification each of the selected segments;

and marking on the low magnification image, each of the segments viewed at high magnification.

10. The method of

claim 1

including the steps of locating the microscope on the Internet having a web site therefor;

including the step of connecting an observer'"'"'s computer and viewing screen over the Internet to the computer-controlled microscope at the web site in real time and transmitting the digitized signals for low magnification image and high magnification image from the web site to the observer'"'"'s computer and viewing screen.

11. The method of

claim 1

including the step of enhancing the reconstructed, high magnification image by color filtering.

12. The method of

claim 1

including the step of providing a full color, low magnification, digitized image to the observer for aiding the observer in selecting a plurality of areas of interest on a priority basis.

13. The method of

claim 1

including the steps of:

analyzing the specimen on the slide with image analysis tests; and

reporting the results of the tests to the observer while at least one of the respective images is being viewed by the observer.

14. The method of

claim 13

wherein the specimen is a biological specimen, and the analyzing step including performing analysis tests on tissue or cells in the biological specimen.

15. A method in accordance with

claim 1

including the step of the observer changing the magnification of the high magnification image to a magnification intermediate the high and low magnification images by the use of an image analysis algorithm.

16. A method in accordance with

claim 1

including the step of storing high magnification image tiles of adjacent sections of the specimen;

and scrolling the high magnification image tiles to bring into view on a screen adjacent sections not previously viewed.

17. A method of acquiring and analyzing biological specimens with a computer-controlled microscope, said method comprising the steps of:

scanning and digitizing a series of fields of views through a microscope objective lens of a low magnification and providing a series of digitized fields of view images;

arranging and positioning the adjacent fields of view images and forming therefrom a composite, macro image in color of the specimen having a field of view larger than that of the objective lens;

displaying the macro color image to the observer to provide the observer with a larger field of view of the specimen than the observer could see through an objective lens on the microscope;

selecting a segment of the specimen from the macro image for viewing at a higher magnification than the low magnification;

scanning and digitizing a series of fields of view through a microscope objective lens of a higher magnification and providing a series of digitized images;

arranging and positioning the adjacent fields of view images at high magnification and forming therefrom a composite micro image of the segment having a field of view larger than that of the high magnification objective lens; and

displaying to the observer both colored composite macro image and the micro image each having a field of view substantially larger than their respective objective lens.

18. A method in accordance with

claim 17

including the step of marking on the macro image, the location of the micro image, so that the observer understands where the micro image is located on the macro image.

19. A method in accordance with

claim 18

including the step of selecting several segments each for viewing as a micro image and marking the location of each micro image on the composite image.

20. A method in accordance with

claim 19

including the step of recording the macro image with the marks thereon to provide a record of the segments viewed by the observer at high magnification to provide an audit trail for a later auditing of the segments viewed at higher resolution.

23. A method in accordance with

claim 20

including the steps of:

digitizing a plurality of fields of view through the microscope at each of the low and high magnifications to form a series of digitized image tiles;

arranging the respective image tiles to form the respective macro and micro images;

displaying to the observer the micro and macro digitized images that each have a larger field of view than a single field of view through respective objective lens for low and high magnification.

21. A method of acquiring and analyzing biological specimens with a computer-controlled microscope;

said method comprising the steps of;

providing a biological specimen in position for scanning through the computer-controlled microscope;

scanning and digitizing the biological specimen through the microscope at a first, low magnification and acquiring and storing a digitized image of the specimen in color;

displaying the stored, low magnification, digitized image in color to an observer to provide a macro image for viewing by the observer;

selecting a segment of the colored, digitized, low magnification image for viewing at a higher magnification than the low magnification;

scanning the segment at the higher magnification and acquiring a colored digitized image of the segment at the higher magnification to provide a micro image of the segment;

making available in real time to the view of the observer, the respective micro and the macro images; and

performing a quantitative analysis of the specimen from the digitized high magnification image.

22. A method in accordance with

claim 21

including the step of marking on the macro image the position of the micro image so that the viewer understands the location of the micro view.

24. A microscopic system for analyzing a specimen and a segment of the specimen, said apparatus comprising:

a microscope having a plurality of objective lens to acquire images at different magnifications;

a computer-controlled light illumination subsystem on the microscope for adjusting light illumination for different magnifications;

a computer-controlled, focusing subsystem on the microscope for adjusting the focus for different magnifications;

a computer-controlled X and Y stage to position the specimen to be viewed at specified X and Y coordinates;

an imaging subsystem connected to a microscope to acquire and digitize the images of the specimen;

a first optical system and image acquisition system having a first objective lens to acquire and arrange a series of images at a low magnification to provide a macro digitized image of the specimen having a field of view larger than the field of view from the objective lens used for the low magnification acquisition of images;

a second optical and image acquisition system having a second, higher magnification objective lens to acquire and arrange a series of images at higher magnification and to provide a micro view image of the segment having a field of view larger than the field of view from the higher magnification objective lens; and

a display screen device to make available in real time the micro and macro images on screens of the screen device.

25. A system in accordance with

claim 24

wherein the first image subsystem includes an X and Y position storage device for storing the X and Y coordinates for the macro image.

26. A system in accordance with

claim 24

wherein the second image acquisition subsystem has a subsystem for reconstructing high magnification image tiles with their X and Y coordinates positioned to reproduce spatially high resolution image tiles into the micro image as if the micro image was in the original image.

27. A system in accordance with

claim 24

wherein the first image acquisition subsystem has a subsystem for reconstructing low resolution image tiles with their X and Y coordinates positioned to reproduce spatially the low resolution tiles into a macro image of the specimen.

28. A method for analyzing biological specimens by an image analysis system having a computer-controlled, automated microscope comprising the following steps:

placing a biological specimen in a microscope for viewing;

using a computer terminal to control the image analysis system and microscope to acquire low magnification, image tiles of the specimen and to provide low magnification, composite, macro image from the plurality of image tiles;

displaying the macro image on a screen of the computer terminal;

interactively selecting at least one point of interest on the displayed macro, specimen image for viewing at a higher magnification;

sending signals from the computer terminal to control the image analysis system and the computer-controlled microscope to acquire a plurality of higher magnification, image tiles and to provide a low magnification, composite micro image; and

analyzing the micro images of the specimen.

29. A method in accordance with

claim 28

including the step of analyzing a layer of tissue cells in the specimen for pre-invasive neoplasia.

30. A method in accordance with

claim 29

including the step of measuring morphometric features of the specimen and measuring texture features of the specimen from the micro images.

31. A method in accordance with

claim 30

wherein the step of interactively selecting a least one point of interest includes the further step of selecting a series of points along a basal layer of tissue in the specimen for analysis at high magnification.

32. A method in accordance with

claim 28

wherein the steps of interactively selecting at least one point of interest includes the further step of selecting a plurality of points of interest on the macro image;

and

recording the location of each selected point of interest.

33. A method in accordance with

claim 28

wherein the step of interactively selecting at least one point of interest comprises the further steps of:

marking a region on the macro image on the screen that includes portions of adjacent, low magnification image tiles;

acquiring a high magnification image tile from each of marked low magnification tiles; and

assembling these high magnification image tiles into a micro image.

34. A method in accordance with

claim 28

including the step of simultaneously displaying on computer windows of the computer both micro and macro images of the specimen.

35. A method in accordance with

claim 28

wherein the computer terminal is at a remote location from the computer-controlled microscope, and including the step of first connecting the computer terminal at the remote location over a transmission channel to the image analysis system and the computer-controlled microscope.