Sample observation device for generating super resolution image

• US 8,780,192 B2
• Filed: 10/03/2013
• Issued: 07/15/2014
• Est. Priority Date: 09/30/2010
• Status: Active Grant

• Current Assignee: Evident Corporation
• Original Assignee: Olympus Corporation
• Inventors: Hayashi, Shinichi
• Primary Examiner(s): Kim, Hee-Yong
• Application Number:
  US14/045,569
• Publication Number:
  US 20140036057A1
• Time in Patent Office: 285 Days
• Field of Search: 348/79
• US Class Current: 348/79
• CPC Class Codes: G02B 21/0032 Optical details of illumination, e.g. light-sources, pinholes, beam splitters, slits, fibers G02B21/0036 - G02B21/008; means for illumination of specimens in general G02B21/06
• CPC Class Codes: G02B 21/0044 moving apertures, e.g. Nipkow disks, rotating lens arrays
• CPC Class Codes: G02B 21/0076 arrangements using fluorescence or luminescence
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A sample observation device includes: an excitation light generation unit; an intermediate image forming unit projecting excitation light to a sample and forming an intermediate image of the sample at an intermediate image position with observing light; a confocal modulation unit modulating spatial intensity distributions of the excitation light and the intermediate image at the position; a modulation drive unit moving a pattern of the modulation unit; an image relay unit relaying on a image forming surface the intermediate image; an image pickup unit converting the distribution of relayed intermediate image into digital image data; and an image processing unit processing on the digital image data. Cutoff frequency of the relay unit and Nyquist frequency of the pickup unit exceed cutoff frequency of the forming unit, and the processing unit performs a high frequency enhancing process for enhancing the high frequency component exceeding the cutoff frequency of the forming unit.

Claims

1. A sample observation device comprising:

an excitation light generation unit which generates excitation light for exciting a sample;

an intermediate image forming unit which projects the excitation light to the sample, and forms an intermediate image of the sample at an intermediate image position from observing light generated by illuminating the sample with the excitation light;

a confocal modulation unit which modulates spatial intensity distributions of the excitation light and the intermediate image at the intermediate image position;

a modulation drive unit which moves a modulation pattern of the confocal modulation unit relative to the intermediate image;

an image relay unit which relays on an image forming surface the intermediate image whose spatial intensity distribution has been modulated;

an image pickup unit which converts the spatial intensity distribution of the intermediate image relayed on the image forming surface into digital image data; and

an image processing unit which performs image processing on the digital image data,

wherein;

a cutoff frequency of the image relay unit exceeds a cutoff frequency of the intermediate image forming unit;

a Nyquist frequency of the image pickup unit exceeds the cutoff frequency of the intermediate image forming unit;

the image processing unit performs a high frequency enhancing process for enhancing a high frequency component exceeding the cutoff frequency of the intermediate image forming unit;

the confocal modulation unit comprises a mask having a pinhole-shaped aperture arranged periodically;

the sample observation device further comprises a microlens array having a condensing position at the pinhole-shaped aperture, wherein the microlens array is arranged between the excitation light generation unit and the confocal modulation unit; and

the modulation drive unit is configured to move the microlens array and the mask without changing relative positions of the microlens array and the mask.

2. The device according to claim 1, wherein:

the modulation pattern is a pattern that repeats cyclically; and

an exposing time for acquiring an image by the image pickup unit is equal to or greater than a time in which the modulation pattern moves one cycle of the modulation pattern with respect to the intermediate image.

3. The device according to claim 1, wherein the cutoff frequency of the image relay unit is equal to or greater than 1.5 times the cutoff frequency of the intermediate image forming unit.

4. The device according to claim 1, wherein the Nyquist frequency of the image pickup unit is equal to or greater than 1.5 times the cutoff frequency of the intermediate image forming unit.

5. The device according to claim 1, wherein a period of the pinhole-shaped aperture arranged periodically is longer than a period regulated by the cutoff frequency of the intermediate image forming unit.

6. The device according to claim 1, wherein:

the confocal modulation unit comprises a rotary disc having a pattern that repeats cyclically; and

the modulation drive unit comprises a motor for rotating the rotary disc.

7. The device according to claim 1, wherein the sample observation device comprises a plurality of confocal modulation units having different modulation patterns, wherein any confocal modulation unit is selectable from among the plurality of confocal modulation units, and insertable into an optical path.

8. The device according to claim 1, wherein the intermediate image forming unit has a variable projection magnification.

9. The device according to claim 1, wherein the image relay unit has a variable relay magnification.

10. The device according to claim 1, wherein the high frequency enhancing process subtracts from the digital image data a pseudo wide-field image obtained by operating a low pass filter on the digital image data.

11. The device according to claim 1, wherein the high frequency enhancing process is a convolution process using a kernel matrix.

12. The device according to claim 1, wherein the high frequency enhancing process subtracts from the digital image data a wide-field image of the sample obtained without operating the confocal modulation unit.

13. A sample observation device comprising:

an excitation light generation unit which generates excitation light for exciting a sample;

an intermediate image forming unit which projects the excitation light to the sample, and forms an intermediate image of the sample at an intermediate image position from observing light generated by illuminating the sample with the excitation light;

a confocal modulation unit which modulates spatial intensity distributions of the excitation light and the intermediate image at the intermediate image position;

a modulation drive unit which moves a modulation pattern of the confocal modulation unit relative to the intermediate image;

an image relay unit which relays on an image forming surface the intermediate image whose spatial intensity distribution has been modulated;

an image pickup unit which converts the spatial intensity distribution of the intermediate image relayed on the image forming surface into digital image data; and

an image processing unit which performs image processing on the digital image data,

wherein;

a cutoff frequency of the image relay unit exceeds a cutoff frequency of the intermediate image forming unit;

a Nyquist frequency of the image pickup unit exceeds the cutoff frequency of the intermediate image forming unit;

the image processing unit performs a high frequency enhancing process for enhancing a high frequency component exceeding the cutoff frequency of the intermediate image forming unit;

the confocal modulation unit is a mask having a slit-shaped aperture arranged periodically;

the sample observation device further comprises a cylindrical lens array having a condensing position at the slit-shaped aperture, wherein the cylindrical lens array is arranged between the excitation light generation unit and the confocal modulation unit; and

the modulation drive unit is configured to move the cylindrical lens array and the mask without changing relative positions of the cylindrical lens array and the mask.

14. The device according to claim 13, wherein a period of the slit-shaped aperture arranged periodically is longer than a period regulated by the cutoff frequency of the intermediate image forming unit.

15. The device according to claim 13, wherein the mask includes a phase shift film.

16. The device according to claim 13, wherein:

the confocal modulation unit comprises a rotary disc having a pattern that repeats cyclically; and

the modulation drive unit comprises a motor for rotating the rotary disc.

17. The device according to claim 13, wherein the sample observation device comprises a plurality of confocal modulation units having different modulation patterns, wherein any confocal modulation unit is selectable from among the plurality of confocal modulation units, and insertable into an optical path.

18. A sample observation device comprising:

an excitation light generation unit which generates excitation light for exciting a sample;

an intermediate image forming unit which projects the excitation light to the sample, and forms an intermediate image of the sample at an intermediate image position from observing light generated by illuminating the sample with the excitation light;

a confocal modulation unit which modulates spatial intensity distributions of the excitation light and the intermediate image at the intermediate image position;

a modulation drive unit which moves a modulation pattern of the confocal modulation unit relative to the intermediate image;

an image relay unit which relays on an image forming surface the intermediate image whose spatial intensity distribution has been modulated;

an image pickup unit which converts the spatial intensity distribution of the intermediate image relayed on the image forming surface into digital image data; and

an image processing unit which performs image processing on the digital image data,

wherein;

a cutoff frequency of the image relay unit exceeds a cutoff frequency of the intermediate image forming unit;

a Nyquist frequency of the image pickup unit exceeds the cutoff frequency of the intermediate image forming unit;

the image processing unit performs a high frequency enhancing process for enhancing a high frequency component exceeding the cutoff frequency of the intermediate image forming unit;

the sample observation device further comprises an annular illumination unit for annularly illuminating the sample with the excitation light, wherein the annular illumination unit comprises a concentric dichroic filter which is located near a position conjugate with a pupil position of an objective lens included in the intermediate image forming unit;

the sample observation device further comprises a transparent object having a refractive index that is higher than a refractive index of a medium surrounding the sample, wherein the transparent object is arranged between the sample and the intermediate image forming unit; and

the concentric dichroic filter restricts an angle of the excitation light which enters an interface between the sample and the transparent object such that more than 90% of the excitation light which has entered the interface is totally reflected.

19. The device according to claim 18, wherein:

the confocal modulation unit comprises a rotary disc having a pattern that repeats cyclically; and

the modulation drive unit comprises a motor for rotating the rotary disc.

20. The device according to claim 18, wherein the sample observation device comprises a plurality of confocal modulation units having different modulation patterns, wherein any confocal modulation unit is selectable from among the plurality of confocal modulation units, and insertable into an optical path.