Apparatus for remote control of a microscope

• US 6,674,884 B2
• Filed: 11/13/2001
• Issued: 01/06/2004
• Est. Priority Date: 08/23/1996
• Status: Expired due to Term

• Current Assignee: Evident Scientific Incorporated (Evident Corporation)
• Original Assignee: Bacus Laboratories Incorporated (Olympus Corporation)
• Inventors: Bacus, James W., Bacus, James V.
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): NAKHJAVAN, SHERVIN K
• Application Number:
  US10/008,519
• Publication Number:
  US 20020061127A1
• Time in Patent Office: 784 Days
• Field of Search: 382/128, 382/129, 382/133, 348/79, 348/80, 396/432, 359/144, 359/369, 359/370, 359/372, 359/379, 359/380, 359/381, 359/382, 359/383, 709/203, 709/212, 709/217, 709/219, 702/19, 725/713
• US Class Current: 382/133
• CPC Class Codes: G01N 15/1433 using image recognition
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06V 20/69 Microscopic objects, e.g. biological cells or cellular parts
• CPC Class Codes: G16H 10/40 for data related to laboratory analysis, e.g. patient specimen analysis
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 40/63 for local operation

Abstract

An apparatus and method acquires and stores multiple resolution images from a specimen on a support and provides to the user a low magnification, reconstructed macro image of the entire specimen, or a large portion thereof, to aid the person in selecting points of interest to be viewed or analyzed at higher magnifications and resolution. The reconstructed image is formed of a large number of tiled, stored images which are coordinated and assembled to form the macro image of the specimen which is displayed on a monitor. Preferably, the stored, reconstructed image is reduced further in size by a software system before it is displayed to the user. The display may be on a local monitor over a local area network or sent over the Internet to the user who is typically a pathologist. The user selects by a marker such as a cursor the defined area of interest or region and then views higher magnification images or has them analyzed. Preferably, the pathologist can scroll to shift digitized, adjacent image tiles into view on the monitor. A fully computer-controlled microscope is used to acquire and store the digitized images and the illustrated microscope can be remotely controlled to change objective lenses, focus, light intensity, filters, field diaphragm, and to shift the microscope stage by a controller.

Claims

1. A method of using a computer-controlled microscope imaging system for acquiring an analyzing areas of interest, said method comprising the steps of:

scanning and digitizing a specimen at a low magnification through a microscope at a first location;

displaying to the observer at a remote location the low magnification digitized image of the specimen;

using a computer terminal at a remote location and selecting a segment of the specimen from a displayed low magnification digitized image for viewing at a higher magnification greater than the low magnification;

sending signals from the computer at the remote location over an Internet or Intranet transmission channel for automatically scanning the selected segment at the higher magnification and digitizing a field of view image of the segment;

sending signals representative of the high magnification field of view image over the Internet or Intranet transmission channel to the remote location; and

making available at the remote location in real time to the view of the observer the respective, low magnification, digitized image and the high magnification, digitized field of view image of the segment.

2. A method for analyzing biological specimens by an image analysis system having a computer-controlled, automated microscope comprising the following steps:

placing a biological specimen in a microscope at a first location for viewing at a remote location;

connecting a computer terminal at a remote location over an Internet or intranet transmission channel;

using a computer terminal at the remote location to control the image analysis system and microscope at the first location to acquire low magnification, adjacent image tiles of the specimen and to provide a low magnification, composite, macro image from the plurality of image tiles with the series of images being in sensing registration at their respective edges;

displaying at the remote location the macro image on a screen of the computer terminal from a transmission over the transmission channel;

interactively selecting at the remote location at least one point of interest on the displayed macro, specimen image for viewing at a higher magnification;

sending signals from the computer terminal to control and to operate the image analysis system and the computer-controlled microscope at the first location to change objective lenses to acquire a plurality of higher magnification, adjacent image tiles and to provide a high magnification, composite micro image therefrom with the series of images being in sensing registration at their respective edges; and

sending signals representative of the higher magnification, image tiles from the first location to the remote location and displaying at the remote location the micro image of the point of interest on the specimen.

3. A method in accordance with claim 2 further comprising analyzing a layer of tissue cells in the specimen for pre-invasive neoplasia.

4. A method in accordance with claim 3 further comprising measuring morphometric features of the specimen and measuring texture features of the specimen from the micro images.

5. A method in accordance with claim 4 wherein interactively selecting at least one point of interest further comprises selecting a series of points along a basal layer of tissue in the specimen for analysis at high magnification.

6. A method in accordance with claim 2 wherein interactively selecting at least one point of interest further comprises:

selecting a plurality of points of interest on the macro image; and

recording the location of each selected point of interest.

7. A method in accordance with claim 2 wherein interactively selecting at least one point of interest further comprises:

marking by the remote user of a region on the macro image on the screen that includes portions of adjacent, low magnification image tiles;

acquiring high magnification image tile for each of marked low magnification tiles; and

assembling these high magnification image tiles into a micro image and displaying them.

8. A method in accordance with claim 2 including:

simultaneously displaying on computer windows of the computer both micro and macro images of the specimen.

9. A method in accordance with claim 2 wherein the computer terminal is at a remote location from the computer-controlled microscope, and comprising:

first connecting the computer terminal at the remote location over a transmission channel to the image analysis system and the computer-controlled microscope which is located only at the first location.

10. A method for analyzing biological specimens by an image analysis system having a computer-controlled, automated microscope comprising the following steps:

placing a biological specimen in a microscope at a first location for viewing at a remote location;

connecting a remote computer terminal at a remote location over an Internet or Intranet transmission channel;

using the remote computer terminal at the remote location to control the image analysis system and microscope at the first location;

sending signals from the remote computer terminal over the transmission channel to control and to operate the image analysis system and microscope at the first location to acquire a plurality of higher magnification, adjacent, digitized image tiles with the series of images being in registration with each other;

storing the digitized image tiles for use with a server at the first computer terminal; and

transmitting by the server from the first location to the remote location and displaying at the remote location images formed from the digitized image tiles transmitted over the transmission channel.

11. A method in accordance with claim 10 comprising:

connecting the server at the first location and the remote computer channel over the Internet transmission channel and sending the stored digitized image tiles over the Internet transmission channel from the first location to the remote location and operating the microscope at the first location by signals transmitted over the Internet transmission channel to the first location.

12. A method in accordance with claim 11 comprising:

providing a consulting pathologist at the remote location for operating the microscope; and

sending the digitized image tiles over the Internet transmission and operating the microscope with a live transmission of digitized images tiles within a period of minutes from start of the use of the image analysis system from the remote location.

13. An apparatus for acquiring and transmitting over an Internet or intranet transmission channel a magnified view of a substantial portion of a biological specimen and a higher magnification view of a selected point of interest for viewing at a location remote from the microscope, the apparatus comprising:

an imaging system for acquiring a magnified digitized image of a substantial portion of the biological specimen and including a microscope to acquire a higher magnification, digitized, field of view images for the selected point of interest of the specimen;

a control system at the remote and first locations operable by an operator at the remote location and over the Internet or intranet transmission channel to select a point of interest from the magnified, digitized image of a substantial portion of the biological specimen and causing the image system and microscope to acquire the higher magnification, digitized image for storing in the storing system for a transmitting of the higher magnification image to the remote location;

a storing system for storing data representing the magnified, digitized image as data for transmission over the Internet or intranet transmission channel and for storing data representing the higher magnification image; and

a display device at the remote location for displaying the transmitted higher magnification image of the selected point of interest from the transmitted data and for displaying the magnified, digitized image of a substantial portion of the biological specimen from the transmitted data.

14. An apparatus in accordance with claim 13 wherein the imaging system for acquiring a magnified digitized image of a substantial portion of the specimen comprises:

a computer controlled microscope stage system for stepping to acquire a series of adjacent, field of view images taken through the microscope and for registration with one another to provide a composite image of adjacent fields of view larger than a single field of view from the microscope.

15. An apparatus in accordance with claim 14 wherein:

an objective lens being used in the microscope having a lower magnification and resolution than an objective lens being used for the high magnification in order to obtain the low magnification, registered adjacent field of view images comprising the composite image.

16. An apparatus in accordance with claim 14 wherein the display device comprises:

a scrolling feature to allow scrolling into view on the display device higher magnification images of adjacent fields of view.

17. An apparatus in accordance with claim 13 wherein the imaging system including a microscope to acquire a higher magnification, digitized field of view image for the selected point of interest comprises:

a computer controlled microscope stage system movable intermittently in steps when acquiring a series of adjacent fields of view taken through the microscope at the higher magnification and registered with one another to provide a composite image of adjacent fields of view larger than a single field of view for the point of interest.

18. An apparatus in accordance with claim 13 wherein the imaging system comprises an X and Y storage device for storing the locations of the images of each of the fields of view.

19. An apparatus in accordance with claim 13 wherein the display device comprises:

a device operable for switching back and forth between digitized higher magnification images at several different magnifications for the same point of interest.

20. An apparatus in accordance with claim 13 wherein the display device includes a monitor having a first portion for displaying the low magnification composite image and a second portion for displaying simultaneously the higher magnification image.

21. An apparatus in accordance with claim 13 comprising:

a marker on the display device movable by the operator to navigate along to composite low magnification image to select a suspect area for viewing by the operator on a subsequent display on the display device.

22. An apparatus for acquiring and transmitting over the Internet magnified microscopic views from a remotely operated microscope, the apparatus comprising:

an image analysis system including a computer controlled microscope for receiving a biological specimen thereon and for acquiring field of view images at different magnifications of the specimen;

a remote computer terminal at a remote location for connection over the Internet to the image analysis system and for operating the image analysis system from the remote location to acquire a high magnification field of view image from the specimen;

a storage device for storing a digitized image of a high magnification field of view as data for transmission;

a server associate with the image analysis system and storage device for transmission of the data to the remote location; and

a display device at the remote location for displaying a low magnification image showing the morphology of the specimen for navigation thereon by the operator when remotely controlling the microscope to select a point of interest and to acquire and transmit a high magnification field of view image of the selected point of interest.

23. An apparatus in accordance with claim 22 comprising:

objective lens in the microscope being positioned to provide multiple magnification images at magnification greater than that used for the low magnification image.

24. A method of performing a quantitative analysis on stored, digitized views of an optically magnified specimen comprising:

storing a plurality of digitized image optically magnified to allow a view of a region and to allow a quantitative measurement thereof a region larger than a single field of view;

selecting a first region of interest from a composite digitized image of at least a portion of the specimen;

performing a quantitative analysis on a digitized image from the first region of interest;

requesting a second stored digitized image from a second region of interest for analysis;

receiving a stored digitized image from the second region of interest;

performing the same quantitative analysis on the stored digitized image front the second region of interest thereby providing a quantitative analysis with respect to the region of interest; and

providing a lower magnification composite view of the specimen to a user to enable the selection of a region of interest for quantitative analysis which is done on a higher magnification and resolution image.

25. A method in accordance with claim 24 comprising:

storing the digitized images at a first location; and

transmitting the stored digitized images to a plurality of other locations to allow simultaneous quantitative analysis to be performed on the same region of interest.

26. A method in accordance with claim 24 comprising:

providing an Internet transmission channel and a server for distribution of the stored displayed images over the transmission channel.

27. A method in accordance with claim 24 comprising:

transmitting the selected digitized images over the Internet to a user.

28. A method in accordance with claim 27 comprising:

a plurality of users accessing the stored, digitized images and a plurality of users receiving their respectively selected digitized images of their respectively selected region of interest for the quantitative analysis.

29. A method of quantitative analysis of a biological specimen comprising:

providing scanned and stored data of optically magnified digitized images from the biological specimen at a first location;

providing an analysis program at a second location physically removed from the first location for performing a quantitative analysis on an optically magnified image;

retrieving data of scanned and stored images upon a request from the first location for transfer to the second request location;

transferring the data for the digitized images to thin second location and storing the retrieved data for optically magnified, digitized images at the second location;

displaying to an observer at the second location a comprehensive image of the specimen formed from the retrieved data;

selecting by the observer of a region of interest from the specimen from the comprehensive view on which to perform the quantitative analysis; and

using the analysis program to perform a quantitative analysis on successive, optical magnified adjacent images from the specimen comprising the region of interest obtained from the retrieved data stored at the second location.

30. A method of analysis of a biological specimen on a microscopic slide comprising:

storing a first digitized overall image larger than a single field of view and formed of adjacent fields of view to provide an optically magnified view at a first magnification of a biological specimen on a microscope slide;

storing a second digitized image of optically of a magnified view from the biological specimen at a higher magnification than the first magnification;

viewing the overall image and selecting therefrom a region of interest tot quantitative analysis from the overall image;

retrieving from storage a selected higher magnification, digitized image for analysis; and

using the selected high magnification digitized image to conduct a quantitative analysis of the selected region of interest.

31. A method in accordance with claim 30 comprising:

retrieving from storage, a plurality of adjacent, higher magnification images comprising the selected region of interest; and

using these higher magnification, retrieved images to conduct quantitative analysis of the selected region of interest.

32. A method in accordance with claim 30 wherein the selected region comprises:

Multiple, adjacent fields of view of the specimen at the higher magnification.

33. A method in accordance with claim 30 comprising:

strolling to shift into view an adjacent high magnification image for viewing by the viewer.

34. A method in accordance with claim 30 wherein the viewer selects a suspicious morphological feature displayed in the overall composite image and navigates therealong when selecting the region of interest to be analyzed.

35. A method in accordance with claim 30 comprising:

displaying simultaneously both the overall image and the higher magnification image; and

displaying a location mark on the overall image to located thereon the region of interest.

36. A method in accordance with claim 30 wherein the retrieving from storage comprises:

sending stored data for the digitized, optically magnified image from a first location over the Internet to a remote location; and

the selecting of the region of interest comprises sending a request over the Internet from a remote location for a selected higher magnification image to be sent from the first location to the remote location over the Internet.

37. A method in accordance with claim 30 wherein the providing of the stored digitized image comprises:

scanning a biological specimen at a first magnification for the composite images; and

scanning the biological specimen a second time at a second higher magnification for the higher magnifications digitized images.

38. A method of viewing suspicious regions of a biological specimen on a microscope slide for analysis comprising:

providing stored data for a digitized optically magnified overall image of the specimen;

providing stored data for digitized images of the specimen at a higher optical magnification than that of the overall image;

retrieving from storage an overall image of the specimen;

viewing the overall image and intelligently navigating therein to select a region of interest for analysis at higher magnification;

marking on the overall image a selected region for analysis at higher magnification;

retrieving from storage higher magnification images of the selected region of interest; and

displaying a higher magnification image of the region for viewing and analysis.

39. A method in accordance with claim 38 compromising

storing the stored data for the optically magnified overall image and the higher magnification image on a hard disk;

retrieving the stored data from the hard disk; and

displaying simultaneously to a viewer both the optically magnified overall image and the higher magnification images of the selected region of interest.

40. A method in accordance with claim 39,

wherein the stored data is at a location remote from the viewing location and the stored data is sent over a communication channel useable by a plurality at users.

41. A method in accordance with claim 40 wherein the common communication channel is the Internet.

42. A method of analysis using a virtual microscope slide comprising:

providing the biological specimen on a microscope slide;

optically magnifying the biological specimen and providing a magnified image;

digitizing the magnified image to provide the virtual microscope slide;

digitally storing the virtual microscope slide to provide a stored virtual microscope slide image of the specimen;

viewing morphological features on the stored virtual microscope slide image;

selecting a region of morphological interest on the retrieved virtual microscope slide image;

retrieving from storage a plurality of adjacent, higher magnification images of the region of interest; and

using the retrieved image high magnification images to conduct an analysis of the morphological feature for the biological specimen.

43. A method in accordance with claim 42 comprising:

sending the stored, digitized image over the Internet to provide a virtual microscope slide image for analysis at a remote location.

44. A method in accordance with claim 42 comprising:

retrieving from storage higher magnification images at different magnifications of the region of morphological interest to allow analysis of a morphological feature at different resolutions.

45. A method in accordance with claim 42, comprising:

displaying images simultaneously of adjacent, higher magnification fields of view for the specimen.