Method and apparatus for creating a virtual microscope slide

• US 20050254696A1
• Filed: 06/21/2004
• Published: 11/17/2005
• Est. Priority Date: 03/03/1997
• Status: Active Grant

• Current Assignee: Evident Scientific Incorporated (Evident Corporation)
• Original Assignee: Bacus Laboratories Incorporated (Olympus Corporation)
• Inventors: Bacus, James W., Bacus, James V.
• Granted Patent: US 8,625,920 B2
• Time in Patent Office: Days
• US Class Current: 382/128
• CPC Class Codes: G01N 15/1433 using image recognition
• CPC Class Codes: G02B 21/367 providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06V 20/69 Microscopic objects, e.g. biological cells or cellular parts
• CPC Class Codes: G16H 10/40 for data related to laboratory analysis, e.g. patient specimen analysis
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 40/60 for the operation of medical equipment or devices
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

A method and apparatus are disclosed for constructing a virtual microscope slide comprised of digitally scanned images from a microscope specimen. The digitally scanned images are arranged in a tiled format convenient for viewing without a microscope, and for transferring the tiled images for viewing by another at a remote location. Several original microscope views at a low magnification are digitized and stored as digitized images coherently seamed together to provide an overall virtual, macro image of the specimen at a lower resolution. Several original microscope views at higher magnifications are digitized and stored as digitized images coherently seamed together to provide virtual micro images at higher resolution. A data structure is formed with these virtual macro and micro digitized images along with their mapping coordinates. Preferably, a generic viewing program is also provided in the data structure that allows remote users to manipulate and interpret the tiled images on the user'"'"'s monitor. Also, the data structure is formed with significantly compressed data so as to be transmitted over low bandwidth channels, such as the Internet, without loss of resolution that would interfere with the analysis at a remotely-located pathologist receiving the data structure over the Internet. The preferred interactive program allows the pathologist to scroll and view neighboring image areas of interest. A marker on the macro image indicates to the user the location of the micro image and assists the user in selecting areas from the macro image to be viewed at higher resolution and magnification.

Claims

1-67. -67. (canceled)

68. A system of programs for creating and viewing a data structure of an image of a specimen on a microscope slide comprising:

a scanning routine for digitally scanning magnified images of contiguous segments of the specimen on a microscope slide and providing data for magnified images of segments of the specimen;

a recording routine for recording the scanned digital image data; and

a linking routine for linking the series of contiguous image segments with a control program effective for viewing, manipulating and reconstructing the image segments.

69. A system of programs in accordance with claim 68 comprising:

a dynamic, self-executing control program.

70. A system of programs in accordance with claim 69 wherein the dynamic, self-executing program is a Java applet.

71. A system of programs in accordance with claim 69 wherein the dynamic, self-executing program is an Active-X applet.

72. The program of claim 68 further comprising a first display routine for displaying a micro image of a portion of the specimen at a first magnification;

and

a second display routine for displaying a macro second image at a lower magnification comprising an overall view from the specimen on the microscope slide.

73. The program of claim 72 further comprising a routine for selecting a point on said macro image and for producing a corresponding micro image at said point.

74. The program of claim 73 further comprising a coordinate display routine for displaying the coordinates of said point to the user.

75. A system in accordance with claim 68 wherein the recording routine comprises:

storing data structures for multiple microscope specimens on a portable medium.

76. A method for viewing virtual microscope slides having a magnified image of substantially an entire virtual microscope slide specimen and higher magnification images of points on the specimen comprising:

providing an Internet or intranet communication channel;

providing one or more receivers connected to the Internet or intranet communication channel to receive virtual microscope slides for viewing;

storing one or more sets of virtual microscope digital slides at a first station connected to the Internet or intranet communication channel;

the virtual microscope slide comprising scanned data from a plurality of adjacent segments of the specimen useable to form a magnified overall image to allow selection of points that are so small on the specimen that magnification is needed to aid in the selection of points, the data for higher magnification images being useable to enlarge the selected point and at sufficient resolution for medical analyzing;

transmitting a stored virtual microscope digital slide over the Internet or intranet communication channel to a receiver requesting a virtual microscope slide or a portion thereof; and

viewing a display of a virtual microscope slide of the overall view of substantially the entire specimen and the higher magnification images of points for medical analysis of the specimen.

77. A method in accordance with claim 76 comprising:

quantifying characteristics of the displayed specimen from a displayed higher magnification image.

78. A method in accordance with claim 76 comprising:

displaying points at higher magnification of a stained or adhesive treated specimen thereby contrasting different characteristics of the specimen being viewed at higher magnification.

79. A method in accordance with claim 76 comprising:

scrolling into view higher magnification images to mimic the changing of objective lens in a microscope.

87. A method in accordance with claim 76 comprising:

transmitting compressed data using a reduced percentage of a corresponding original microscope image.

88. A method in accordance with claim 76 comprising:

transmitting a stored low-magnification thumbnail view of a specimen on a microscope slide over the Internet or Intranet communication channel to the receiving station to assist in locating areas for viewing, and transmitting and displaying a higher magnification, higher resolution digital images of an area of the specimen.

89. A method in accordance with claim 76 wherein the transmitting of stored virtual microscope slides comprises:

transmitting a stored set of low-magnification, digitized image segments and spatially aligning and displaying said segments as the thumbnail view of the specimen representation.

90. A method in accordance with claim 76 comprising:

simultaneously displaying low-magnification digitized image and high-magnification digitized image coordinated to one another by an addressable coordinate system.

91. A method in accordance with claim 76 comprising:

scrolling a portion of an adjacent image into view on a display device.

92. A method in accordance with claim 76 comprising:

providing a plurality of first stations each capable of transmitting virtual microscope slides over the Internet or Intranet communication channel; and

addressing one of the plurality of stations and requesting a desired virtual microscope slide therefrom.

93. A method in accordance with claim 76 comprising:

selecting from a plurality of magnifications one of the higher magnifications for viewing a portion of a displayed virtual microscope slide as a higher resolution image.

94. A method in accordance with claim 93 comprising:

providing an addressable coordinate system for the stored images at different magnifications so that each of the higher magnification images can be easily located with respect to a lower magnification image.

95. A method in accordance with claim 76 comprising:

providing images corresponding to a single field of view of an objective lens of a microscope.

96. A method in accordance with claim 76 comprising:

transmitting from;

the first station to a receiving station a control program for, effecting the displaying, manipulating and reconstructing of the image segments to form the virtual microscope slide at the receiving station.

97. A method in accordance with claim 96 wherein the transmitting from the first station to a receiving station of a control program wherein the central program comprises:

a self-executing control program; and

transferring a stored digital slide with the self-executing control program for effecting the displaying, manipulating and reconstructing of the specimens.

98. A method in accordance with claim 76 comprising:

transmitting a self-executing data structure from the first station having stored digital image segments and a dynamic, self-executing program for effecting the viewing, reconstructing and manipulating of the stored digital image segments.

99. A method in accordance with claim 98 comprising:

transmitting stored micro and macro images as part of the data structure; and

transmitting an addressable coordinate system for the micro and macro images as part of the data structure.

100. A method in accordance with claim 99 comprising:

flipping back and forth between the high resolution micro images and macro images during an analysis of the specimen representation on the virtual microscope slide.

101. A method in accordance with claim 76 wherein the storing of one or more sets of virtual microscope digital slides at the first station comprises:

having stored on a portable medium microscope slide images of specimens to be viewed by medical students through the viewing of virtual digital slides; and

making the virtual digital slides on the portable recording medium available to multiple medical students at multiple locations.

80. A method in accordance with claim 80 comprising:

using a computer and its display device for displaying the overall and higher magnification images; and

using a marker on the displayed image to navigate within the overall image to select points for viewing at higher magnifications.

81. A method in accordance with claim 80 comprising:

using data for image tiles to form and to display a magnified image which is larger than a single field-of-view obtained through magnifying lens used when capturing the image data being used for this displayed, magnified image.

82. A method in accordance with claim 81 comprising:

assembling digitized image data of the digital image segments of substantially the entire specimen, the higher magnification images and mapping coordinates therefor to define a data structure; and

transmitting the data structure over the Internet or Intranet communication channel for use in assembling and displaying digitized images of the specimen.

83. A method in accordance with claim 82 comprising:

spatially aligning transmitted digital image segments by using information transmitted from the first station to form the virtual microscope slide.

84. A method in accordance with claim 83 comprising:

spatially aligning the transmitted digital image segments on a display screen of a display device.

85. A method in accordance with claim 83 comprising:

forming an overall, composite image of digital image segments at the first station; and

then transmitting the composite image and displaying the virtual microscope slide at the requesting receiver.

86. A method in accordance with claim 83 comprising:

transmitting addressable coordinate information over the Internet or Intranet communication channel to the receiving station for use in aligning of the digital image segments.

102. A method for viewing virtual microscope slides having a magnified image of substantially an entire virtual microscope slide specimen and multiple higher magnification images of points on the specimen comprising:

providing an Internet or intranet communication channel;

providing one or more receivers connected to the Internet or intranet communication channel to receive virtual microscope slides for viewing;

storing one or more sets of virtual microscope digital slides having an overall image formed of adjacent segments of the specimen and points thereof at high magnification at a first station connected to the Internet or intranet communication channel;

transmitting a stored virtual microscope digital slide over the Internet or intranet communication channel to a receiving station requesting a virtual microscope slide or a portion thereof; and

displaying an overall image of the specimen and displaying a point on the specimen at multiple higher magnifications.

103. A method of constructing and using a data structure of images from a small specimen on a microscope slide comprising:

digitally scanning a specimen and acquiring digital image data of a series of digitized adjacent image segments of the specimen useable to provide an overall magnified image of the specimen and higher magnification images of selected points on the specimen;

storing the scanned digital image data;

transferring the scanned, digital image data over an Internet or intranet communication channel;

providing a control program for the data structure for viewing the overall magnified image and higher magnification images of selected points on the specimen;

selecting a point on said overall magnified image; and

displaying from the stored digital image data a corresponding higher magnification image of the selected point.

104. The method of claim 103 further comprising:

displaying a first image comprising a portion of the specimen as an overall macro view; and

displaying a second image comprising a higher magnification view from the specimen on the microscope slide at a higher magnification than the magnification of the overall view to allow an analysis of disease or a quantification of characteristics of the specimen.

105. The method of claim 103 wherein said image segments are stored as bit-mapped files.

106. The method of claim 105 further comprising converting said bit-mapped files to JPEG files.

107. The method of claim 104 comprising:

zooming between higher magnification images of the same point on the specimen.

108. A method in accordance with claim 103 comprising:

transferring the digital image data to a web browser.

109. A method in accordance with claim 108 comprising:

transferring of the digital image data over a dynamic intranet.

110. A method in accordance with claim 103 comprising:

retoggling between higher magnification images at respectively different magnifications and stored on the web browser.

111. A method in accordance with claim 103 comprising:

scrolling a portion of an image being viewed in a direction to cause the image being viewed to shift to include, in a new image, a portion of the image from a neighboring image that was not previously viewed in the last image viewed by a user.

112. A method in accordance with claim 103 comprising:

making the data structure when initially scanning and storing the image data.

113. A method in accordance with claim 103 comprising:

making and storing a data structure at the web server.

114. A method in accordance with claim 103 comprising:

transferring the data structure and control program to a browser for viewing of the overall image.

115. A method in accordance with claim 114 wherein the control program comprises:

a Java applet.

116. A method in accordance with claim 114 wherein the control program comprises:

an Active-X applet.

117. A method in accordance with claim 114 wherein the control program is previously installed at the browser and comprising:

transferring the data structure over a common communication channel to the browser for use with the previously installed control program.

118. A method in accordance with claim 103 comprising:

performing a quantitative analysis on the displayed images.

119. A method of constructing and using a virtual microscope data structure of images from a specimen on a microscope slide comprising:

digitally scanning individual contiguous segments of a specimen through a, microscope lens and acquiring image data for magnified images of the specimen on a microscope slide useable to create an overall magnified image of the specimen or a portion thereof and useable to create magnified images of a point on the specimen at a higher magnification than the magnification of the overall image;

storing the scanned digital image data;

transferring the scanned, digital image data over an Internet or intranet communication channel;

providing a control program for the data structure for viewing the overall magnified image of the specimen and the respective first and second higher magnification views of the point; and

using the control program to allow viewing simultaneously of the overall image of the specimen area and a higher magnification image of the point.

120. A method in accordance with claim 119 comprising:

digitally scanning individual contiguous segments at a first magnification to acquire image data for the overall image; and

digitally scanning a second time at a higher magnification to acquire the image data for the higher magnification images of the specimen.

121. A method of local viewing of magnified images from a specimen on a microscope slide comprising:

storing all of the digitized, image data of a virtual microscope slide at a storage medium located at the local display device of a virtual microscope slide to display an overall magnified image of the specimen at a low magnification and multiple magnifications images of points on the specimen;

displaying the overall image and simultaneously displaying a higher magnification image of a point on the specimen from the stored digitized data; and

using a control program at the local station for viewing the overall magnified image and for selecting higher magnification images for a point for local viewing.

122. A method in accordance with claim 121 comprising:

flipping back and forth between a plurality of higher magnification images of the point of the specimens.

123. A method in accordance with claim 121 comprising:

selecting a region of the overall image for display at high magnification; and

displaying a higher magnification image for the selected region.

124. A method in accordance with claim 121 comprising:

scrolling the display higher magnification to view adjacent portions of the specimen.

125. A method in accordance with claim 121 wherein the control program comprises a browser.

126. A method in accordance with claim 121 wherein the providing of a control program comprises:

providing a dynamic self-executing program as the control program.

127. A method in accordance with claim 126 comprising:

transferring the magnified images and the dynamic self-executing program as a data structure over a common communication channel.

128. A method in accordance with claim 126 wherein the self-executing program comprises a Java applet.

129. A method in accordance with claim 126 wherein the self-executing program comprises an Active-X applet.

130. A method in accordance with claim 126 wherein the self-executing program comprises an Internet browser.

131. A method in accordance with claim 121 comprising:

displaying the higher magnification images at sufficient optical resolution for analysis of stained or otherwise treated characteristics of the specimen.

132. A method in accordance with claim 121 comprising:

quantifying characteristics of the treated specimen from the higher magnification and resolution of a point on the specimen.