Sample observation device for generating super resolution image

• US 8,817,088 B2
• Filed: 09/22/2011
• Issued: 08/26/2014
• Est. Priority Date: 09/30/2010
• Status: Active Grant

• Current Assignee: Evident Corporation
• Original Assignee: Olympus Corporation
• Inventors: Hayashi, Shinichi
• Primary Examiner(s): Kim, Hee-Yong
• Application Number:
  US13/240,236
• Publication Number:
  US 20120081535A1
• Time in Patent Office: 1,069 Days
• Field of Search: 348/79
• US Class Current: 348/79
• CPC Class Codes: G02B 21/0032 Optical details of illumination, e.g. light-sources, pinholes, beam splitters, slits, fibers G02B21/0036 - G02B21/008; means for illumination of specimens in general G02B21/06
• CPC Class Codes: G02B 21/0044 moving apertures, e.g. Nipkow disks, rotating lens arrays
• CPC Class Codes: G02B 21/0076 arrangements using fluorescence or luminescence
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A sample observation device includes: an excitation light generation unit; an intermediate image forming unit projecting excitation light to a sample and forming an intermediate image of the sample at an intermediate image position with observing light; a confocal modulation unit modulating spatial intensity distributions of the excitation light and the intermediate image at the position; a modulation drive unit moving a pattern of the modulation unit; an image relay unit relaying on a image forming surface the intermediate image; an image pickup unit converting the distribution of relayed intermediate image into digital image data; and an image processing unit processing on the digital image data. Cutoff frequency of the relay unit and Nyquist frequency of the pickup unit exceed cutoff frequency of the forming unit, and the processing unit performs a high frequency enhancing process for enhancing the high frequency component exceeding the cutoff frequency of the forming unit.

Claims

1. A sample observation device comprising:

an excitation light generation unit which generates excitation light for exciting a sample;

an intermediate image forming unit which projects the excitation light to the sample, and forms an intermediate image of the sample at an intermediate image position from observing light generated by illuminating the sample with the excitation light;

a confocal modulation unit which modulates spatial intensity distributions of the excitation light and the intermediate image at the intermediate image position;

a modulation drive unit which moves a modulation pattern of the confocal modulation unit relative to the intermediate image;

an image relay unit which relays on an image forming surface the intermediate image whose spatial intensity distribution has been modulated;

an image pickup unit which converts the spatial intensity distribution of the intermediate image relayed on the image forming surface into digital image data; and

an image processing unit which performs image processing on the digital image data,

wherein;

the confocal modulation unit comprises a mask having a pinhole-shaped aperture arranged periodically;

a cutoff frequency of the image relay unit exceeds a cutoff frequency of the intermediate image forming unit;

a Nyquist frequency of the image pickup unit exceeds the cutoff frequency of the intermediate image forming unit; and

the image processing unit performs a high frequency enhancing process for enhancing a high frequency component exceeding the cutoff frequency of the intermediate image forming unit.

2. The device according to claim 1, wherein:

the modulation pattern is a pattern that repeats cyclically; and

an exposing time for acquiring an image by the image pickup unit is equal to or greater than a time in which the modulation pattern moves one cycle of the modulation pattern with respect to the intermediate image.

3. The device according to claim 1, wherein the cutoff frequency of the image relay unit is equal to or greater than 1.5 times the cutoff frequency of the intermediate image forming unit.

4. The device according to claim 1, wherein the Nyquist frequency of the image pickup unit is equal to or greater than 1.5 times the cutoff frequency of the intermediate image forming unit.

5. The device according to claim 1, wherein a period of the pinhole-shaped aperture arranged periodically is longer than a period regulated by the cutoff frequency of the intermediate image forming unit.

6. The device according to claim 1, wherein:

the confocal modulation unit comprises a rotary disc that comprises the mask; and

the modulation drive unit comprises a motor for rotating the rotary disc.

7. The device according to claim 6, wherein the rotary disc is a Nipkow disc.

8. The device according to claim 1, wherein:

the confocal modulation unit comprises a plate that comprises the mask; and

the modulation drive unit comprises a vibrator for vibrating the plate.

9. The device according to claim 1, wherein:

the confocal modulation unit comprises a rotary drum that comprises the mask; and

the modulation drive unit comprises a motor for rotating the rotary drum.

10. The device according to claim 1, wherein the sample observation device comprises a plurality of confocal modulation units having different modulation patterns, wherein any confocal modulation unit is selectable from among the plurality of confocal modulation units, and insertable into an optical path.

11. The device according to claim 1, wherein:

the confocal modulation unit comprises a spatial light modulator;

the modulation drive unit comprises a drive circuit for driving the spatial light modulator.

12. The device according to claim 1, wherein the intermediate image forming unit has a variable projection magnification.

13. The device according to claim 1, wherein the image relay unit has a variable relay magnification.

14. The device according to claim 1, wherein the high frequency enhancing process subtracts from the digital image data a pseudo wide-field image obtained by operating a low pass filter on the digital image data.

15. The device according to claim 14, wherein the high frequency enhancing process is a convolution process using a kernel matrix.

16. The device according to claim 1, wherein the high frequency enhancing process subtracts from the digital image data a wide-field image of the sample obtained without operating the confocal modulation unit.

17. A sample observation device comprising:

an excitation light generation unit which generates excitation light for exciting a sample;

an intermediate image forming unit which projects the excitation light to the sample, and forms an intermediate image of the sample at an intermediate image position from observing light generated by illuminating the sample with the excitation light;

a rotary disc in which a mask having a line-and-space structure arranged periodically is formed, and which modulates spatial intensity distributions of the excitation light and the intermediate image at the intermediate image position;

a motor which rotates the rotary disc to move a modulation pattern of the rotary disc relative to the intermediate image;

an image relay unit which relays on an image forming surface the intermediate image whose spatial intensity distribution has been modulated;

an image pickup unit which converts the spatial intensity distribution of the intermediate image relayed on the image forming surface into digital image data; and

an image processing unit which performs image processing on the digital image data,

wherein;

an exposing time for acquisition of one image by the image pickup unit is equal to or greater than a time needed for a half rotation of the rotary disc;

a cutoff frequency of the image relay unit exceeds a cutoff frequency of the intermediate image forming unit;

a Nyquist frequency of the image pickup unit exceeds the cutoff frequency of the intermediate image forming unit; and

the image processing unit performs a high frequency enhancing process for enhancing a high frequency component exceeding the cutoff frequency of the intermediate image forming unit.

18. The device according to claim 17, wherein a period of the line-and-space structure is longer than a period regulated by the cutoff frequency of the intermediate image forming unit.

19. The device according to claim 17, wherein the mask includes a phase shift film.

20. The device according to claim 17, further comprising:

an annular illumination unit for annularly illuminating the sample with the excitation light,

wherein the annular illumination unit is located near a position conjugate with a pupil position of an objective included in the intermediate image forming unit.