Method and apparatus for creating a virtual microscope slide

• US 6,272,235 B1
• Filed: 02/27/1998
• Issued: 08/07/2001
• Est. Priority Date: 03/03/1997
• Status: Expired due to Term

• Current Assignee: Evident Scientific Incorporated (Evident Corporation)
• Original Assignee: Bacus Laboratories Incorporated (Olympus Corporation)
• Inventors: Bacus, James W., Bacus, James V.
• Primary Examiner(s): Black, Thomas
• Assistant Examiner(s): Veillard, Jacques
• Application Number:
  US09/032,514
• Time in Patent Office: 1,257 Days
• Field of Search: 382/133, 382/128, 382/132, 382/284, 382/294, 382/305, 382/308, 707/513, 707/102, 707/104, 702/21
• US Class Current: 382/133
• CPC Class Codes: G01N 15/1433 using image recognition
• CPC Class Codes: G02B 21/367 providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06V 20/69 Microscopic objects, e.g. biological cells or cellular parts
• CPC Class Codes: G16H 10/40 for data related to laboratory analysis, e.g. patient specimen analysis
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 40/60 for the operation of medical equipment or devices
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

A method and apparatus are disclosed for constructing a virtual microscope slide comprised of digitally scanned images from a microscope specimen. The digitally scanned images are arranged in a tiled format convenient for viewing without a microscope, and for transferring the tiled images for viewing by another at a remote location. Several original microscope views at a low magnification are digitized and stored as digitized images coherently seamed together to provide an overall virtual, macro image of the specimen at a lower resolution. Several original microscope views at higher magnifications are digitized and stored as digitized images coherently seamed together to provide virtual micro images at higher resolution. A data structure is formed with these virtual macro and micro digitized images along with their mapping coordinates. Preferably, a generic viewing program is also provided in the data structure that allows remote users to manipulate and interpret the tiled images on the user'"'"'s monitor. Also, the data structure is formed with significantly compressed data so as to be transmitted over low bandwidth channels, such as the Internet, without loss of resolution that would interfere with the analysis at a remotely-located pathologist receiving the data structure over the Internet. The preferred interactive program allows the pathologist to scroll and view neighboring image areas of interest. A marker on the macro image indicates to the user the location of the micro image and assists the user in selecting areas from the macro image to be viewed at higher resolution and magnification.

Claims

1. A method of constructing and using a data structure of images from a specimen on a microscope slide using a microscope having an objective lens comprising:

digitally scanning and storing images from the specimen on a microscope slide to create a plurality of individual contiguous image tiles at a resolution finer than the optical resolution of the objective lens of the microscope and with a digital spacing between pixels finer than the optical resolution of the objective lens;

providing a control program for the data structure for viewing, manipulating and reconstructing the image tiles; and

transferring the scanned, digital image tiles over an Internet or intranet communication channel; and

using the control program to allow viewing of a digital reconstructed composite image formed from contiguous image tiles of a substantially larger image area than the area of the individually acquired tiles.

2. The method of claim 1 further comprising providing the optical resolution for the recorded images of at least about 0.2 microns at 40×

.

3. The method of claim 2 further comprising:

displaying a first image comprising a portion of the specimen as an overall macro view;

displaying a second image comprising higher resolution view from the specimen on the microscope slide at a higher magnification than the magnification of the overall macro view;

selecting a point on said overall macro image with a marker; and

producing a corresponding higher magnification image at the location of the marker.

4. The method of claim 3 further comprising displaying the X-Y coordinates of said point selected on the macro image.

5. A method in accordance with claim 3 wherein the step of selecting a point includes:

moving a marker to a desired point on the overall image; and

executing a command to produce the composite, high magnification and high resolution image from the selected point.

6. The method of claim 1 wherein said image tiles are stored as bit-mapped files.

7. The method of claim 6 further comprising converting said bit-mapped files to JPEG files.

8. The method of claim 1, comprising scanning, storing and displaying of said composite images of contiguous images from the area of interest of said specimen of at least three magnifications.

9. A method in accordance with claim 1 including the step of transferring the digital images to a web browser.

10. A method in accordance with claim 9 including the step of retoggling between lower magnification images and higher magnification images stored on the web browser.

11. A method in accordance with claim 1 comprising:

providing a dynamic self-executing program as the control program;

transferring the data structure having the digital image tiles and the dynamic, self-executing program over the Internet or Intranet communication channel.

12. A method in accordance with claim 1 comprising:

scanning and storing multiple magnification and multiple resolution images and creating image tiles having multiple magnifications and multiple resolutions;

transferring the image tiles of multiple magnifications and multiple resolutions over the Internet or Intranet communication channel to allow viewing of lower magnification and higher magnification images;

providing an addressable coordinate system so that the higher magnification images can be easily located with respect to the lower magnification images; and

toggling back and forth between displays of the lower and higher magnification images by the viewer when analyzing these images from the specimen.

13. A method in accordance with claim 1 including the step of:

scrolling a portion of an image being viewed in a direction to cause the image being viewed to shift to include, in a new image, a portion of the image from a neighboring tiled image that was not previously viewed in the last image viewed by a user.

14. A method in accordance with claim 1 comprising:

storing slide specimen data structures on the portable recording media and making them available to medical students.

15. A method in accordance with claim 1 comprising:

the storing of the slide specimens comprises storing them on CD-ROMs; and

the CD-ROMs are placed on a web server and are accessible by medical students at various times.

16. The method of claim 1 comprising:

using a microscope stage with a plurality of increments of stage movement for a single pixel dimension of a scanner used for the digital scanning and a display used for displaying the composite reconstructed images.

17. A system of programs for creating and viewing a data structure of images using a microscope with an objective lens from a specimen on a microscope slide comprising:

a scanning routine for digitally scanning the images from the specimen on a microscope slide at a resolution finer than the optical resolution of the objective lens of the microscope and with a digital spacing between pixels finer than the optical resolution of the objective lens;

a recording routine for recording the scanned digital images in a series of contiguous image tiles; and

a linking routine for linking the series of contiguous image tiles with a control program effective for viewing, manipulating and reconstructing the image tiles.

18. The program of claim 17 further comprising a first display routine for displaying a micro image comprising a portion of the scanned image at a first magnification and a second display routine for displaying a macro second image comprising an overall view from the specimen on the microscope slide.

19. The program of claim 17 further comprising a routine for selecting a point on said macro image and for producing a corresponding micro image at said point.

20. The program of claim 17 further comprising a coordinate display routine for displaying the coordinates of said point to the user.

21. The program of claim 17 wherein the routine comprises:

storing the data structure for multiple microscope specimens on a portable medium for display to medical students; and

making these data structures on the portable medium available to medical students at various times.

22. The program of claim 21 wherein the routine comprises:

providing a web server accessible by medical students for providing access by the students to the specimen images stored as data structures on the portable recording medium.

23. An apparatus for creating and using a data structure comprising:

a computer-controlled microscope imaging system for digitally scanning multiple tiled images from an area of interest from a specimen on a microscope support at a first resolution and image magnification;

a program and a microscope stage and positioning system for shifting the stage for recording the scanned digital images in a series of contiguous image tiles at a resolution finer than the optical resolution of the objective lens of the microscope and with a digital spacing between pixels finer than the optical resolution of the objective lens; and

a program for linking the series of contiguous image tiles with a control program effective for viewing, manipulating and reconstructing the image tiles.

24. An apparatus in accordance with claim 23 wherein the optical resolution for the recorded images is at least about 0.2 microns at 40×

.

25. An apparatus in accordance with claim 24 comprising:

an Internet communication channel connected to the apparatus for transmitting the compressed data over the Internet channel to a remote location.

26. An apparatus in accordance with claim 23 wherein the computer controlled microscope scans and records tile images at a lower magnification and resolution which are combined into an overall composite image for the area;

and

an addressable coordinate system provides addresses for the respective low and high magnification image tiles so that images can be seamed together and the higher magnification, tiled images can be easily located with respect to the lower magnification, tiled images.

27. An apparatus in accordance with claim 26 further comprising:

a dedicated viewer for viewing the images and for flipping back and forth between the low resolution overall, composite image and the high resolution composite images.

28. An apparatus in accordance with claim 26 including a marker program to mark an addressable area on the overall image and to cause the addressed area to appear at a higher resolution composite image.

29. An apparatus in accordance with claim 23 wherein a program for scrolling allows the user to scroll a portion of a neighboring image into view.

30. An apparatus in accordance with claim 23 including an address display to display the coordinates to assist multiple viewers to identify the same area for analysis and commentary.

31. An apparatus in accordance with claim 23 comprising:

a microscope stage having more increments of stage movement than a single pixel dimension of a scanner used for digital scanning and a display used for displaying the composite image.

32. An apparatus in accordance with claim 23 for viewing data structures by students comprising:

a program for storing a plurality of specimens as scanned digital contiguous tiles on a portable medium;

transferring the portable medium to a student teaching location; and

making the stored image tiles available to the students at the student teaching location.

33. A method for viewing a portion of or an entire virtual microscope slide having a specimen representation comprised of sets of digitized image tiles, the method comprising:

providing an Internet or intranet communication channel;

providing one or more receivers connected to the Internet or intranet communication channel to receive virtual microscope slides for viewing;

storing one or more sets of virtual microscope digital slides at a first station connected to the Internet or intranet communication channel;

transmitting stored digital image tiles at a first resolution and magnification over the Internet or intranet communication channel to a receiver requesting a virtual microscope slide or a portion thereof; and

providing a display of a virtual microscope slide formed from the transmitted set of digital image tiles as a composite reconstructed view formed of selected contiguous image tiles at the first resolution and magnification of the original optical image and providing a display of an overall view of an area of interest of the specimen representation at a lower resolution and magnification at the requesting receiver.

34. A method in accordance with claim 33 comprising:

assembling the digitized image tiles and mapping coordinates therefor to define a data structure; and

transmitting the data structure over the Internet or Intranet communication channel for use in assembling and displaying the digitized image tiles.

35. A method in accordance with claim 33 comprising:

spatially aligning the transmitted digital image tiles by using information transmitted from the first station to form the virtual microscope slide.

36. A method in accordance with claim 35 comprising:

spatially aligning the transmitted digital tiles on a display screen of a display device by displaying sequentially adjacent tiles on the display screen.

37. A method in accordance with claim 35 comprising:

forming a composite image of digital image tiles at the first station; and

then transmitting the composite image and displaying the virtual microscope slide to the requesting receiver.

38. A method in accordance with claim 33 comprising:

transmitting addressable coordinate information over the Internet or Intranet communication channel to the receiving station for use in aligning of the digital image tiles.

39. A method in accordance with claim 38 comprising:

transmitting compressed data using a reduced percentage of a corresponding original microscope image.

40. A method in accordance with claim 33 comprising:

transmitting a stored low-magnification thumbnail view of a specimen on a microscope slide over the Internet or Intranet communication channel to the receiving station to assist in locating areas for viewing of the transmitted and displayed higher resolution digital image tiles for the virtual microscope slide.

41. A method in accordance with claim 33 comprising:

transmitting a stored set of low-magnification, digitized image tiles and spatially aligning and displaying said tiles as the thumbnail view of the specimen representation.

42. A method in accordance with claim 33 comprising:

simultaneously displaying low-magnification digitized, image tiles and high-magnification digitized, image tiles coordinated to one another by an addressable coordinate system.

43. A method in accordance with claim 33 comprising:

scrolling a portion of adjacent image tiles into view on a display device.

44. A method in accordance with claim 33 comprising:

providing a plurality of first stations each capable of transmitting virtual microscope slides over the Internet or Intranet communication channel; and

addressing one of the plurality of stations and requesting a desired virtual microscope slide therefrom.

45. A method in accordance with claim 33 comprising:

selecting from a plurality of magnifications one of the higher magnifications for viewing a portion of a displayed virtual microscope slide as a higher resolution composite image.

46. A method in accordance with claim 45 comprising:

providing an addressable coordinate system for the stored, image tiles at different magnifications so that each of the higher magnification images can be easily located with respect to a lower magnification image.

47. A method in accordance with claim 33 comprising:

providing image tiles corresponding to a single field of view of an objective lens of a microscope.

48. A method in accordance with claim 33 comprising:

transmitting from the first station to a receiving station a control program for effecting the displaying, manipulating and reconstructing of the image tiles to form the virtual microscope slide at the receiving station.

49. A method in accordance with claim 48 wherein the transmitting from the first station to a receiving station of a control program comprises:

transferring a stored digital slide with a self-executing control program for effecting the displaying, manipulating and reconstructing of the image tiles to form the virtual microscope slide at the receiving station.

50. A method in accordance with claim 33 comprising:

transmitting a self-executing data structure from the first station comprising stored digital image tiles and a dynamic, self-executing program for effecting the viewing, reconstructing and manipulating of the stored digital image tiles.

51. A method in accordance with claim 50 comprising:

transmitting stored micro and macro image tiles as part of the data structure; and

transmitting an addressable coordinate system for the micro and macro image tiles as part of the data structure.

52. A method in accordance with claim 51 comprising:

flipping back and forth between the high resolution micro images and macro image tiles during an analysis of the specimen representation on the virtual microscope slide.

53. A method in accordance with claim 33 wherein the storing of one or more sets of virtual microscope digital slides at the first station comprises:

having stored on a portable medium microscope slide images of specimens to be viewed by medical students through the viewing of virtual digital slides; and

making the virtual digital slides on the portable recording medium available to multiple medical students at multiple locations.

54. A method of viewing microscope slide specimens for educational or testing purposes comprising:

providing portable media with stored digitally scanned images of contiguous fields of view from a specimen on a microscope slide of an area of interest larger than a single high resolution image;

providing a viewing program for manipulating the digitally scanned images to edge align the contiguous images into reconstructed composite images at one or more reconstructed magnification levels; and

using the viewing program to allow the display of composite images and to select areas on the specimen for viewing and to switch back and forth between composite images displayed on a display device at different resolutions and magnifications.

55. A method in accordance with claim 54 wherein the educational purpose is the teaching of medical, dental or veterinary students.

56. A method in accordance with claim 54 wherein the educational or testing purpose is for proficiency testing in pathology;

or advanced education of pathologists, dentists, veterinarians or other medical professionals.

57. A method in accordance with claim 54 wherein the testing purpose is the Board Certification of Pathologists.

58. A method in accordance with claim 54 wherein the control program comprises:

a self-executing program for use to manipulate the scanned, stored images for viewing.

59. A method of viewing of high resolution images from a specimen on a microscope slide comprising:

providing a medium with digitally stored images of contiguous fields of view of an area of interest larger then several contiguous fields of view;

providing high resolution images stored at a digital spacing between adjacent pixels in a specimen image plane being greater than the microscope objective lens optical resolution spacing to preserve the optical resolution for a reconstructed high resolution composite image;

providing edge alignment data for spatial reconstruction of contiguous fields of view into a reconstructed composite image larger than several contiguous fields of view; and

using a control program for viewing, manipulating and reconstructing contiguous images to form the reconstructed composite views with the preserved high resolution.

60. A method in accordance with claim 59 comprising:

providing a preserved optical resolution for the displayed reconstructed, composite image better than the resolution of a single, objective lens viewing the entire specimen area of interest in one field of view.

61. A method in accordance with claim 59 comprising:

providing an optical resolution for the high resolution, stored images of at least about 0.2 microns at 40×

.

62. A method in accordance with claim 59 comprising:

using a medium having stored digitally scanned images of contiguous fields of view at lower and higher resolutions and magnifications of the same area of interest;

selecting a point on the lower resolution and lower magnification and displaying a corresponding higher magnification, reconstructed composite image of the selected point.

63. A method in accordance with claim 59 comprising:

providing the data structure with stored, digitally scanned, contiguous images of contiguous fields of view at the lower magnification and resolution and recorded alignment data for aligning the contiguous images into a composite reconstructed image.

64. A method in accordance with claim 59 comprising:

storing a higher resolution, center portion of the optical image taken through an objective lens of the microscope; and

discarding the outer lower resolution fuzzy portion of the optical image seen through the objective lens.

65. A method in accordance with claim 59 wherein the data structure includes a self-executing program.

66. A method in accordance with claim 59 comprising:

scrolling into view adjacent image tiles thereby simulating the movement of a microscope slide'"'"'s travel to view adjacent points of the specimen.

67. A method of using a data structure to display for analysis an area of interest from a microscope specimen with a view larger than a single field of view, the method comprising:

providing a data structure with stored, digitally scanned, contiguous images on a medium with the images having a first magnification and resolution from the area of interest on the specimen;

providing the data structure with edge alignment data for aligning the contiguous images to form reconstructed composite images larger than a single field of view at the first magnification and resolution;

displaying on a display device an overall image of the area of interest at a magnification and resolution lower than the first magnification and resolution;

selecting a point on the overall image to be displayed; and

displaying the selected point as a corresponding reconstructed, composite image using the edge aligned images to form a reconstructed image which is larger than a single field of view for analysis of the selected point by a viewer.

68. A method in accordance with claim 67 comprising:

scrolling into view on the display device adjacent, contiguous images thereby simulating to the viewer views similar to those achieved when shifting the microscope stage and specimen thereon to view adjacent areas on the specimen.

69. A method in accordance with claim 67:

providing a control program for use with the data structure;

providing a plurality of magnifications and resolution images greater than the overall view'"'"'s magnification and resolution; and

using the plurality of magnifications and resolution images and the control program to display corresponding reconstructed composite images of selected points with each composite image being larger than a single field of view for the corresponding magnification.

70. A method in accordance with claim 67 comprising:

storing the overall view on the recording medium.

71. A method in accordance with claim 67 wherein the storage medium is selected from the group of a CD-ROM or a Jazz drive.

72. A method in accordance with claim 67 comprising:

recreating and displaying the real optical resolution of the original images as viewed through the microscope.

73. A method in accordance with claim 67 comprising:

using a physical X,Y position of the point on the specimen through X,Y coordinates of the microscope state and pixel locations of the point on the display device to assemble and edge align tiled images to form the reconstructed, composite image.

74. A method in accordance with claim 67 comprising:

storing each high resolution image with spacing between adjacent pixels greater than the microscope objective lens optical resolution spacing to ensure a high resolution digital image.

75. A method in accordance with claim 67 comprising:

storing at optical resolution of approximately 0.2 microns for a 40×

magnification, high resolution image.

76. A method in accordance with claim 67 comprising:

providing stored images taken by a microscope having a stage moved through a plurality of increments of stage movement for a single pixel dimension of a scanner used for digital scanning and a display used for displaying the composite reconstructed images.