DUAL-MODE STEREO IMAGING SYSTEM FOR TRACKING AND CONTROL IN SURGICAL AND INTERVENTIONAL PROCEDURES

• US 20130274596A1
• Filed: 04/16/2013
• Published: 10/17/2013
• Est. Priority Date: 04/16/2012
• Status: Active Application

• Current Assignee: Children's National Medical Center
• Original Assignee: Children's National Medical Center
• Inventors: Krieger, Axel, AZIZIAN, Mahdi, Kim, Peter C.W., Leonard, Simon, Shademan, Azad
• Application Number:
  US13/863,954
• Publication Number:
  US 20130274596A1
• Time in Patent Office: Days
• US Class Current: 600/424
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2090/3941 Photoluminescent markers
• CPC Class Codes: A61B 2090/395 with marking agent for marking skin or other tissue
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/32 operating autonomously
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 5/0036 including treatment, e.g., using an implantable medical device, ablating, ventilating
• CPC Class Codes: A61B 5/0071 by measuring fluorescence emission
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/061 Determining position of a probe within the body employing means separate from the probe, e.g. sensing internal probe position employing impedance electrodes on the surface of the body A61B1/00158 takes precedence
• CPC Class Codes: A61B 5/7405 using sound
• CPC Class Codes: A61B 5/7425 Displaying combinations of multiple images regardless of image source, e.g. displaying a reference anatomical image with a live image geometric transforms for image registration G06T3/14; enhancement using image fusion G06T5/50; algorithms for registration of images G06T7/30
• CPC Class Codes: A61B 5/7455 characterised by tactile indication, e.g. vibration or electrical stimulation
• CPC Class Codes: A61M 31/005 for contrast media
• CPC Class Codes: A61M 5/007 for contrast media

Abstract

System and method for tracking and control in medical procedures. The system including a device that deploys fluorescent material on at least one of an organ under surgery and a surgical tool, a visual light source, a fluorescent light source corresponding to an excitation wavelength of the fluorescent material, an image acquisition and control element that controls the visual light source and the fluorescent light source, and captures and digitizes at least one of resulting visual images and fluorescent images, and an image-based tracking module that applies image processing to the visual and fluorescent images, the image processing detecting fluorescent markers on at least one of the organ and the surgical tool.

Claims

1. A system for tracking and control in medical procedures, the system comprising:

a device configured to deploy fluorescent material on at least one of an organ under surgery and a surgical tool;

a visual light source;

a fluorescent light source corresponding to an excitation wavelength of the fluorescent material;

an image acquisition and control element configured to control the visual light source and the fluorescent light source, and configured to capture and digitize at least one of resulting visual images and fluorescent images; and

an image-based tracking module configured to apply image processing to the visual and fluorescent images, the image processing detecting fluorescent markers on at least one of the organ and the surgical tool.

2. The system of claim 1, further comprising:

a surgical robot; and

a visual servoing control module configured to receive tracking information from the image-based tracking module and to control the surgical robot, based on the tracking information, to perform a surgical operation.

3. The system of claim 2, further comprising:

a manual control module configured to enable manual control of the surgical robot in place of control by the visual servoing control module.

4. The system of claim 2, wherein the visual servoing control module is further configured to receive manual input and to control the surgical robot, based on the manual input, to perform a surgical operation.

5. The system of claim 1, further comprising:

a surgical robot; and

a manual control module configured to receive manual input and execute master-slave control of the surgical robot.

6. The system of claim 1, further comprising:

a display configured to display at least one of the visual images and the fluorescent images.

7. The system of claim 1, wherein the image-based tracking module further identifies the organ or the surgical tool based on the detected fluorescent markers.

8. The system of claim 1, wherein the image acquisition and control element further comprises:

a dynamic tunable filter configured to alternatively pass visual light and light emitted by the fluorescent material, and

a charged coupled device configured to capture at least one of visual images and fluorescent images.

9. The system of claim 6, wherein the display is stereoscopic or monoscopic.

10. The system of claim 1, wherein the image acquisition and control element generates stereoscopic or monoscopic images.

11. The system of claim 6, wherein the stereoscopic display is further configured to display visual images and a color coded overlay of fluorescent images.

12. The system of claim 6, wherein the stereoscopic display is further configured to display an augmented reality image by overlaying target points detected by the image-based tracking module.

13. The system of claim 1, wherein the system is configured to provide at least one of visual, audio, and haptic feedback to a system operator, based on information provided by the image-based tracking module.

14. The system of claim 1, wherein the system is configured to operate in each of a manual mode, a semi-autonomous mode, and an autonomous mode.

15. The system of claim 1, wherein image-based tracking module identifies virtual boundaries based on the detected fluorescent markers to designate critical structures.

16. The system of claim 15, further comprising:

a detection device configured to determine whether a surgical tool has passed a boundary and to provide constraints on motion or provide alarms when the boundary has been crossed in order to protect the critical structures.

17. The system of claim 1, wherein the fluorescent light source is a near-infrared (NIR) light source.

18. The system of claim 1, wherein the device that deploys the fluorescent material is configured to deploy the fluorescent material by spraying, painting, attachment, tissue injection, or intravenous injection.

19. A method for performing a medical procedure, the method comprising the steps of:

deploying fluorescent material on at least one of an organ under surgery and a surgical tool;

illuminating the organ, the surgical tool, or both, with a visual light source and a fluorescent light source, the fluorescent light source corresponding to an excitation wavelength of the fluorescent material;

capturing and digitizing images resulting from the illumination by the visual light source and the fluorescent light source; and

applying image processing to the digitized images, the image processing detecting fluorescent markers on at least one of the organ and the surgical tool.

20. The method according to claim 19, further comprising:

generating tracking information by tracking the organ, the surgical tool, or both based on the detected fluorescent markers.

21. The method of claim 19, further comprising:

controlling a surgical robot, based on the tracking information, to perform a surgical operation.

22. The method of claim 21, further comprising:

receiving manual input; and

controlling the surgical robot, based on the manual input, to perform the surgical operation.

23. The method of claim 19, further comprising:

receiving manual input; and

executing master-slave control of a surgical robot based on the on manual input.

24. The method of claim 19, further comprising:

providing a stereoscopic or monoscopic display of the digitized images.

25. The method of claim 19, wherein the step of capturing and digitizing images further comprises generating stereoscopic or monoscopic images.

26. The method of claim 24, further comprising:

displaying visual images and a color coded overlay of fluorescent images.

27. The method of claim 24, further comprising:

displaying an augmented reality image by overlaying target points detected by the image-based tracking module.

28. The method of claim 19, further comprising:

providing at least one of visual, audio, or haptic feedback to a system operator, based on the tracking information.

29. The method of claim 19, further comprising:

identifying the organ or the surgical tool based on the detected fluorescent markers.

30. The method of claim 19, further comprising:

performing a surgical procedure based on the detected fluorescent markers.

31. The method of claim 19, further comprising:

designating critical structures by identifying virtual boundaries based on the detected fluorescent markers.

32. The method of claim 31, further comprising:

determining whether a surgical tool has passed a boundary and providing constraints on motion or providing alarms when the boundary has been crossed in order to protect the critical structures.

33. A system for tracking and control in medical procedures, the system comprising:

means for deploying fluorescent material on at least one of an organ under surgery and a surgical tool;

a visual light source;

a fluorescent light source corresponding to an excitation wavelength of the fluorescent material;

means for controlling the visual light source and the fluorescent light source;

means for capturing and digitizing at least one of resulting visual images and fluorescent images; and

means for applying image processing to the visual and fluorescent images, the image processing detecting fluorescent markers on at least one of the organ and the surgical tool.