Dynamically Controlling an Imaging Microscopy System

• US 20130271575A1
• Filed: 03/15/2013
• Published: 10/17/2013
• Est. Priority Date: 04/11/2012
• Status: Abandoned Application

• Current Assignee: zSpace Inc.
• Original Assignee: zSpace Inc.
• Inventors: Ullmann, Peter F.
• Application Number:
  US13/838,571
• Publication Number:
  US 20130271575A1
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: H04N 13/20 Image signal generators
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes

Abstract

System and method for controlling an imaging microscopy system (IMS). A control module may be coupled to an IMS configured to capture an image of a specified region of a specimen based on a specified perspective by controlling the specimen'"'"'s position and/or orientation relative to an image capture subsystem of the IMS corresponding to the specified perspective. A 6 DOF tracker may detect position and/or orientation of a 6 DOF object with respect to a display of the IMS corresponding to a perspective for image capture of the specimen, and send indicative information thereof to the control module, which may determine the specified perspective based on the information, and may determine the specified region of the specimen for image capture based on the specified perspective. The control module may send information indicating the specified region and perspective to the IMS, thereby controlling capture of the image by the IMS.

Claims

1. An imaging microscopy control system comprising:

a control module, coupled to an imaging microscopy system, wherein the imaging microscopy system is configured to capture an image of a specified region of a specimen staged physical specimen based on a specified perspective by controlling the specimen'"'"'s position and/or orientation relative to an image capture subsystem of the imaging microscopy system corresponding to the specified perspective;

a 6 degree of freedom (DOF) tracking device, coupled to the control module, and configured to;

detect position and/or orientation of a 6 DOF object with respect to a display device of the imaging microscopy system, wherein the position and/or orientation of the 6 DOF object corresponds to a perspective for image capture of the specimen; and

send information indicating the detected position and/or orientation of the 6 DOF object to the control module;

wherein the control module is configured to;

determine the specified perspective based on the information indicating the detected position and/or orientation;

determine the specified region of the physical specimen for image capture based on the specified perspective; and

send information indicating the specified region and the specified perspective to the imaging microscopy system, thereby controlling capture of the image by the image capture subsystem of the imaging microscopy system based on the specified region and the specified perspective;

wherein the information indicating the specified region and the specified perspective is useable by the imaging microscopy system to capture an image of the specimen and to display the image on the display device.

2. The imaging microscopy control system of claim 1,

wherein the image of the specified region of the specimen staged physical specimen comprises a stereo image;

wherein the display device comprises a stereo display device;

wherein the image capture subsystem comprises a stereo image capture subsystem; and

wherein said controlling capture of the image comprises controlling capture of the stereo image by the stereo image capture subsystem of the imaging microscopy system based on the specified region and the specified perspective.

3. The imaging microscopy control system of claim 2, further comprising:

the stereo display device, coupled to the imaging microscopy system, and configured to display the stereo image.

4. The imaging microscopy control system of claim 3, wherein the stereo display device comprises:

a first display, configured to display the stereo image based on the specified perspective; and

a second display, configured to display the stereo image according to another perspective that is different than the specified perspective.

5. The imaging microscopy control system of claim 2, wherein the detected position and/or orientation comprises position and orientation.

6. The imaging microscopy control system of claim 2, wherein a first subset of the 6 DOFs of the 6 DOF tracking device correspond to the detected position and/or orientation, and wherein a second subset of the DOFs of the 6 DOF tracking device correspond to one or more auxiliary control parameters for the image capture subsystem, wherein the one or more auxiliary control parameters comprise one or more of:

magnification level of the imaging microscopy system;

focal plane of the imaging microscopy system;

one or more scanning parameters;

wherein the 6 DOF tracking device is further configured to detect values of the second subset of the DOFs of the 6 DOF tracking device and send information indicating the detected values to the control module;

wherein the control module is configured to;

determine the specified perspective based on the first subset of the 6 DOFs of the 6 DOF tracking device;

determine the one or more auxiliary control parameters based on the detected values of the second subset of the DOFs; and

determine the specified region based on the specified perspective and the one or more auxiliary control parameters corresponding to the second subset of the DOFs of the 6 DOF tracking device.

7. The imaging microscopy control system of claim 2, wherein the 6 DOF tracking device comprises a head tracking device.

8. The imaging microscopy control system of claim 2, wherein the 6 DOF object comprises one or more of:

a user'"'"'s head;

the user'"'"'s eyes;

one or more of the user'"'"'s hands

one or more of the user'"'"'s fingers;

or

a hand-held stylus.

9. The imaging microscopy control system of claim 2, wherein the 6 DOF tracking device comprises a hand held direct interaction device.

10. The imaging microscopy control system of claim 2, said controlling the specimen'"'"'s position and/or orientation relative to an image capture subsystem of the imaging microscopy system comprises controlling one or more of:

position and/or orientation of the specimen stage;

position and/or orientation of one or more sensors of the image capture subsystem;

incident beam geometry of the imaging microscopy system;

or

position and/or orientation of a microscope scan head of the imaging microscopy system with respect to the specimen stage, wherein the specimen stage is stationary.

11. The imaging microscopy control system of claim 2, wherein the 6 DOF tracking device is further configured to:

detect at least one subsequent position and/or orientation of the 6 DOF object with respect to the stereo display device of the imaging microscopy system, wherein the at least one subsequent position and/or orientation of the 6 DOF object corresponds to at least one subsequent perspective for image capture of the specimen; and

send information indicating the detected at least one subsequent position and/or orientation of the 6 DOF object to the control module;

wherein the control module is further configured to;

determine at least one subsequent specified perspective based on the information indicating the detected at least one subsequent position and/or orientation;

determine at least one subsequent specified region of the physical specimen for stereo image capture based on the at least one subsequent specified perspective; and

send information indicating the at least one subsequent specified region and the at least one subsequent specified perspective to the imaging microscopy system, thereby controlling capture of at least one subsequent stereo image by the image capture subsystem of the imaging microscopy system based on the at least one subsequent specified region and the at least one subsequent specified perspective, thereby implementing real time navigation with respect to the specimen.

12. The imaging microscopy control system of claim 11, wherein the specified perspective is a first oblique perspective and wherein the at least one subsequent specified perspective is a second oblique perspective.

13. The imaging microscopy control system of claim 2,

wherein the position and/or orientation of the 6 DOF object is determined using camera triangulation.

14. The imaging microscopy control system of claim 2, wherein the display device comprises an obliquely positioned display.

15. The imaging microscopy control system of claim 2, wherein the imaging microscopy system utilizes one or more of multi-spectrum light, laser, electron beams, or ion beams to image the specimen.

16. The imaging microscopy control system of claim 2,

wherein capture of the stereo image by the image capture subsystem of the imaging microscopy system comprises capture of a stereo pair of images for display on the stereo display device;

wherein the control module is further configured to;

provide a specified interpupillary distance (IPD) that defines a spatial separation between two stereo views corresponding to the stereo pair of images for viewing by a user, thereby controlling capture of the stereo pair of images in accordance with the determined IPD.

17. The imaging microscopy control system of claim 16, wherein the control module is configured to adjust the IPD based on a specified magnification level of the image capture subsystem.

18. The imaging microscopy control system of claim 16, wherein the control module is configured to control capture of the stereo pair of images such that the captured stereo pair of images share a common parallax plane.

19. The imaging microscopy control system of claim 2, wherein capture of the stereo image by the image capture subsystem of the imaging microscopy system comprises capture of a stereo pair of images for display on the stereo display device, and wherein the specimen stage of the imaging microscopy system comprises a eucentric stage.

20. The imaging microscopy control system of claim 2, wherein capture of the stereo image by the image capture subsystem of the imaging microscopy system comprises capture of a stereo pair of images for display on the stereo display device, and wherein the image capture subsystem is configured to capture the stereo pair of images concurrently.

21. The imaging microscopy control system of claim 2, wherein capture of the stereo image by the image capture subsystem of the imaging microscopy system comprises capture of a stereo pair of images for display on the stereo display device, and wherein the image capture subsystem is configured to capture the stereo pair of images consecutively.

22. The imaging microscopy control system of claim 21, wherein the imaging microscopy system is configured to:

utilize an electron or ion beam to image the specimen; and

deflect the electron or ion beam using scan coils to shift the center of the raster scan from a first position whereby a first image of the stereo pair of images is captured, to a second position whereby a second image of the stereo pair of images is captured.

23. A method for controlling an imaging microscopy system, comprising:

providing a control module, coupled to an imaging microscopy system, wherein the imaging microscopy system is configured to capture an image of a specified region of a specimen staged physical specimen based on a specified perspective by controlling the specimen'"'"'s position and/or orientation relative to an image capture subsystem of the imaging microscopy system corresponding to the specified perspective;

providing a 6 degree of freedom (DOF) tracking device, coupled to the control module;

detecting, via the 6 DOF tracking device, position and/or orientation of a 6 DOF object with respect to a display device of the imaging microscopy system, wherein the position and/or orientation of the 6 DOF object corresponds to a perspective for image capture of the specimen; and

sending, by the 6 DOF tracking device, information indicating the detected position and/or orientation of the 6 DOF object to the control module;

determining, by the control module, the specified perspective based on the information indicating the detected position and/or orientation;

determining, by the control module, the specified region of the physical specimen for image capture based on the specified perspective; and

sending, by the control module, information indicating the specified region and the specified perspective to the imaging microscopy system, thereby controlling capture of the image by the image capture subsystem of the imaging microscopy system based on the specified region and the specified perspective, wherein the information indicating the specified region and the specified perspective is useable by the imaging microscopy system to capture an image of the specimen; and

displaying the image on the display device.

24. A non-transitory computer accessible memory medium that stores program instructions executable by a processor to control an imaging microscopy system, wherein the imaging microscopy system is configured to capture an image of a specified region of a specimen staged physical specimen based on a specified perspective by controlling the specimen'"'"'s position and/or orientation relative to an image capture subsystem of the imaging microscopy system corresponding to the specified perspective, wherein to control an imaging microscopy system, the program instructions are executable to implement:

receiving information from a 6 degree of freedom (DOF) tracking device, wherein the information indicating a detected position and/or orientation of a 6 DOF object with respect to a display device of the imaging microscopy system, wherein the position and/or orientation of the 6 DOF object corresponds to a perspective for image capture of the specimen; and

determining the specified perspective based on the information indicating the detected position and/or orientation;

determining the specified region of the physical specimen for image capture based on the specified perspective; and

sending information indicating the specified region and the specified perspective to the imaging microscopy system, thereby controlling capture of the image by the image capture subsystem of the imaging microscopy system based on the specified region and the specified perspective;

wherein the information indicating the specified region and the specified perspective is useable by the imaging microscopy system to capture an image of the specimen and to display the image on the display device.