Method and apparatus for acquiring and reconstructing magnified specimen images from a computer-controlled microscope

• US 6,101,265 A
• Filed: 03/03/1997
• Issued: 08/08/2000
• Est. Priority Date: 08/23/1996
• Status: Expired due to Term

• Current Assignee: Evident Scientific Incorporated (Evident Corporation)
• Original Assignee: Bacus Laboratories Incorporated (Olympus Corporation)
• Inventors: Bacus, James W., Bacus, James V.
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): Le, Hieu C.
• Application Number:
  US08/805,856
• Time in Patent Office: 1,254 Days
• Field of Search: 382/133, 382/134, 382/171, 382/128, 382/240, 382/305, 382/284, 382/294, 364/413.02, 356/372, 345/435
• US Class Current: 382/133
• CPC Class Codes: G01N 15/1433 using image recognition
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06V 20/695 Preprocessing, e.g. image segmentation
• CPC Class Codes: Y10S 128/922 including image analysis

Abstract

An apparatus and method acquires and stores multiple resolution images from a specimen on a support and provides to the user a low magnification, reconstructed macro image of the entire specimen, or a large portion thereof, to aid the person in selecting points of interest to be viewed or analyzed at higher magnifications and resolution. The reconstructed image is formed of a large number of tiled, stored images which are coordinated and assembled to form the macro image of the specimen which is displayed on a monitor. Preferably, the stored, reconstructed image is reduced further in size by a software system before it is displayed to the user. The display may be on a local monitor over a local area network or sent over the Internet to the user who is typically a pathologist. The user selects by a marker such as a cursor the defined area of interest or region and then views higher magnification images or has them analyzed. Preferably, the pathologist can scroll to shift digitized, adjacent image tiles into view on the monitor. A fully computer-controlled microscope is used to acquire and store the digitized images and the illustrated microscope can be remotely controlled to change objective lenses, focus, light intensity, filters, field diaphragm, and to shift the microscope stage by a controller.

Claims

1. A method of using a computer-controlled microscope imaging system for acquiring areas of interest in a specimen larger than a single field of view, said method comprising the steps of:

scanning and digitizing a specimen at a low magnification through a microscope to provide to the user a macro image of the specimen;

displaying to the observer the low magnification, macro image of the specimen;

interactively selecting by the viewer a segment of the specimen from the low magnification, macro image being seen by the viewer and using the selected segment to direct a region of the specimen to be acquired for display at a higher magnification greater than the low magnification;

automatically scanning the selected segment of the specimen at the higher magnification to provide a plurality of spatially adjacent, single fields of view, higher magnification images tiles stored with information to align edges of contiguous image tiles and their reconstruction as a reconstructed, composite, spatially contiguous image larger than a single field of view at higher magnification; and

making available to the view of the observer the respective, low magnification, macro image and the higher magnification, reconstructed composite digitized image of the segment.

2. A method in accordance with claim 1 further comprising:

providing a low magnification, composite image of substantially the entire specimen for reviewing; and

reducing the reviewed magnification of the reconstructed composite, spatially contiguous, composite image before displaying the reconstructed composite image of substantially the entire specimen to the viewer.

3. A method in accordance with claim 1 in which interactively selecting by the viewer of a defined segment of interest further comprises positioning a marker on the low magnification, macro image at a point of interest to be viewed at high magnification and providing to the observer an image at high magnification of the point of interest.

4. A method in accordance with claim 3 wherein:

the high and low magnification images are each reconstructed from image tiles each having a spatially known location stage coordinates from the microscope; and

using the stage coordinates to assure their proper positioning of image tiles in the reconstructed image.

5. The method of claim 3 including the step of displaying the low magnification image of the specimen and displaying a marked segment on the low magnification image, and simultaneously displaying the high magnification image of the marked segment.

6. The method of claim 1 comprising:

shifting and marking different segments and viewing at high magnification each of the selected, marked segments; and

marking on the low magnification image, each of the segments viewed at high magnification.

7. The method of claim 1 comprising:

locating the microscope on the Internet having an Internet address therefor;

including the step of connecting an observer'"'"'s computer and viewing screen over the Internet to control the computer-controlled microscope and transmitting the digitized signals for low magnification, reconstructed, image acquisition and high magnification, reconstructed, composite image acquisition from a web site to the observer'"'"'s computer and viewing screen.

8. The method of claim 1 comprising:

enhancing the reconstructed, high magnification image by color filtering.

9. The method of claim 1 comprising:

providing a full color, low magnification, digitized image to the observer for aiding the observer in selecting a plurality of areas of interest on a priority basis.

10. The method of claim 8 wherein the specimen is a biological specimen, and the analyzing step further comprises performing analysis tests on tissue or cells in the biological specimen.

11. The method of claim 1 comprising:

analyzing the specimen on the slide with image analysis tests; and

reporting the results of the tests to the observer while at least one of the respective images is being viewed by the observer.

12. A method in accordance with claim 1 further comprising:

making and storing high magnification image tiles of adjacent sections of the specimen to form the reconstructed macro image; and

scrolling the high magnification image tiles to bring into view on a screen adjacent sections not previously viewed.

13. A method in accordance with claim 1 comprising:

the observer changing the magnification of the high magnification image to a magnification intermediate between the high and low magnification images by the use of an image analysis algorithm.

14. A method of viewing digitized images biological specimens acquired from a computer-controlled microscope, comprising:

scanning and digitizing a first series of adjacent fields of view through a microscope objective lens of a low magnification and acquiring stage coordinate information for aligning edges of adjacent fields of view when displayed, and providing a first series of digitized adjacent field of view images;

arranging and positioning the adjacent field of view images and using the stage coordinate information forming therefrom a composite, macro image in color of the specimen having a field of view larger than that of the objective lens;

scanning and digitizing a second series of adjacent fields of view through a microscope objective lens from the same microscope but of a higher magnification and acquiring stage coordinate information for aligning edges of adjacent fields of view when displayed, and providing a second series of adjacent digitized field of view images;

arranging and positioning the adjacent field of view images at high magnification using the stage coordinate information and forming therefrom a composite, colored micro image of the segment having a field of view larger than that of the high magnification objective lens; and

displaying to the observer both the colored composite macro image and the colored micro image each having a field of view substantially larger than its respective objective lens.

15. A method in accordance with claim 14 of viewing over the Internet or on an intranet including the step of marking on the macro image, the location of the micro image, so that another observer at another Internet or intranet location understands where the micro image is located on the macro image.

16. A method in accordance with claim 15 further comprising selecting several segments each for viewing as a micro image and marking the location of each micro image on the composite image.

17. A method in accordance with claim 16 further comprising recording the macro image with the marks associated therewith to provide a record of the segments viewed by the observer at high magnification to provide an audit trail for a later auditing of the segments viewed at higher resolution.

18. A method of acquiring and analyzing biological specimens, comprising:

providing a biological specimen in position for scanning through the computer-controlled microscope;

scanning and digitizing adjacent optical fields of view of at least a large portion of the biological specimen through the microscope at a first, low magnification and acquiring and storing digitized images of the adjacent optical fields of view from the specimen in color;

displaying the stored, low magnification, digitized images in sensing registration at their respective edges and in color to an observer to provide a macro image for viewing by the observer;

selecting a segment of the colored, digitized, low magnification image for viewing at a higher magnification than the low magnification;

scanning the segment and digitizing adjacent fields of view at a higher magnification and reconstructing therefrom a colored digitized, micro image of the portion of the segment at the higher magnification;

displaying the higher magnification colored digitized images in sensing registration at their respective edges of the segment at the higher magnification to provide a micro image of the segment;

making available to the view of the observer simultaneously, the respective micro and the macro images; and

the viewer shifting back and forth between the macro and micro images in the course of performing a pathological analysis of the specimen from the digitized high magnification micro image.

19. A method in accordance with claim 18 comprising:

marking on the macro image the position of the micro image so that the view understands the location of the micro view.

20. A method in accordance with claim 18 further comprising:

digitizing a plurality of fields of view through the microscope at each of the low and high magnifications to form a series of digitized image tiles;

arranging the respective image tiles to form the respective macro and micro images; and

displaying to the observer the micro and macro digitized images that each have a larger field of view than a single field of view through respective objective lens for low and high magnification.

21. A microscopic system for analyzing a specimen and a segment of the specimen, said apparatus comprising:

a microscope having a plurality of objective lens to acquire images at different magnifications;

a computer-controlled light illumination subsystem on the microscope for adjusting light illumination for different magnifications;

a computer-controlled, focusing subsystem on the microscope for adjusting the focus for different magnifications;

a computer-controlled X and Y stage system having a closed loop to position the specimen to be viewed at specified X and Y coordinates;

an imaging subsystem connected to a microscope to acquire and digitize the images of the specimen;

a first optical system and image acquisition and storing system having a first objective lens to acquire, arrange, and store a series of adjacent images at a low magnification to provide a composite, reconstructed, macro digitized image of the specimen having a field of view larger than the field of view from the objective lens used for the low magnification acquisition of images with the series of images and information for sensing registration of the images at their respective edges;

a second optical and image acquisition and storing system having a second, higher magnification objective lens to acquire, arrange, and store a series of adjacent images at higher magnification and to provide a composite, reconstructed, micro view image of the segment having a field of view larger than the field of view from the higher magnification objective lens with the series of images and information for sensing registration of the images at their respective edges;

a display screen device to display the composite micro or composite macro images on the display screen device, each of said composite macro and micro images having their respective series of images in sensing registration at their respective edges; and

an X and Y storage device for storing locations of the series of images and information for sensing registration at their respective edges.

22. A system in accordance with claim 21 wherein the second image acquisition subsystem has a subsystem for reconstructing high magnification image tiles with their X and Y coordinates positioned to reproduce spatially high resolution image tiles into the micro image as if the micro image was in the original image.

23. A system in accordance with claim 21 wherein the first image acquisition subsystem has a subsystem for reconstructing low resolution image tiles with their X and Y coordinates positioned to reproduce spatially the low resolution tiles into a macro image of the specimen.