Systems and methods for tracking robotically controlled medical instruments

• US 9,014,851 B2
• Filed: 03/15/2013
• Issued: 04/21/2015
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Auris Health, Inc. (Johnson & Johnson)
• Original Assignee: Hansen Medical Incorporated (Johnson & Johnson)
• Inventors: Wong, Serena, Walker, Sean, Hsu, Jason, Park, June, Tanner, Neal, Murthy, Kiran
• Primary Examiner(s): MANCHO, RONNIE M
• Application Number:
  US13/835,698
• Publication Number:
  US 20140276937A1
• Time in Patent Office: 767 Days
• Field of Search: 600/407, 600/104, 600/423, 600/167, 600/1, 600/114, 600/424, 600/439, 600/478, 600/117, 601/1, 700/245
• US Class Current: 700/245
• CPC Class Codes: A61B 17/00 Surgical instruments, devices or methods, e.g. tourniquets A61B18/00 takes precedence; contraceptive devices, pessaries, or applicators therefor A61F6/00; eye surgery A61F9/007; ear surgery A61F11/00
• CPC Class Codes: A61B 2017/00212 using remote controls
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/2059 Mechanical position encoders
• CPC Class Codes: A61B 2034/2061 using shape-sensors, e.g. fiber shape sensors with Bragg gratings
• CPC Class Codes: A61B 2034/301 for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes
• CPC Class Codes: A61B 2090/3735 Optical coherence tomography [OCT]
• CPC Class Codes: A61B 2090/3782 transmitter or receiver in catheter or minimal invasive instrument
• CPC Class Codes: A61B 2090/3966 Radiopaque markers visible in an X-ray image
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 90/39 Markers, e.g. radio-opaque or breast lesions markers
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 7/248 involving reference images or patches

Abstract

Systems and methods are described herein for tracking an elongate instrument or other medical instrument in an image.

Claims

1. A robotic system for manipulating a tool with respect to a 3D model of a blood vessel, where the tool comprises a shape sensor coupled thereto, the system comprising:

a robotic drive system comprising at least one actuator, the robotic drive system configured to couple with the tool to position the tool with respect to the 3D model, wherein the shape sensor on the tool senses a shape of the tool while in the blood vessel, and wherein the sensed shape of the tool in the blood vessel is correlated against features of the blood vessel in the 3D model; and

a controller configured to compute a registration between a sensor reference frame and a 3D model reference frame such that the controller can direct the robotic drive system in the 3D model reference frame using the registration.

2. The robotic system of claim 1, wherein the controller is further configured to combine a plurality of discrete registrations to produce a combined registration between the sensor reference frame and the 3D model reference frame such that the controller can produce a plurality of signals to direct the robotic drive system in the 3D model reference frame using the combined registration.

3. The robotic drive system of claim 2, further comprising a master input device coupled to the controller, where the master input device uses the combined registration to permit intuitive driving of the tool using the robotic drive system.

4. The robotic drive system of claim 2, wherein the plurality of discrete registrations include at least a shape sensor to tool registration.

5. The robotic drive system of claim 2, wherein the plurality of discrete registrations comprise registering between a group selected from the shape sensor to tool, the tool to the 3D model, the tool to an operating room structure, the tool to the robotic drive system, the robotic drive system to the operating room structure, and the operating room structure to the 3D model.

6. The robotic drive system of claim 4, wherein the shape sensor to tool registration is specific to the tool.

7. The robotic drive system of claim 1, wherein the tool includes at least one optical fiber coupled thereto and where a shape sensor to tool registration is fixed.

8. The robotic drive system of claim 7, wherein the controller is further configured with an algorithm to match a curved shape of the tool determined by the fiber with a corresponding shape in an image.

9. The robotic drive system of claim 8, wherein the algorithm is one of an automated geometric search and a mathematical optimization technique.

10. The robotic drive system of claim 8, wherein the image is one of a pre-operative anatomical image and an intra-operative image.

11. The robotic drive system of claim 1, wherein the tool includes at least one electro-magnetic sensor coupled thereto and where a shape sensor to tool registration is fixed.

12. The robotic drive system of claim 2, wherein the plurality of discrete registrations includes a tool to robotic drive system registration.

13. The robotic drive system of claim 2, wherein the 3D model reference frame comprises a fluoroscopic model of the blood vessel, comprising a plurality of fluoroscopic images.

14. The robotic drive system of claim 13, wherein the plurality of discrete registrations includes a robotic drive system to fluoroscopic imaging device registration.

15. The robotic drive system of claim 13, wherein the plurality of discrete registrations includes a fluoroscopic imaging device to fluoroscopic image registration.

16. The system of claim 1, wherein the 3D model is derived from an imaging device.

17. The robotic drive system of claim 1, wherein the 3D model reference frame comprises the blood vessel in real time.

18. The robotic drive system of claim 2, wherein the plurality of discrete registrations further includes a robotic drive system to table registration and a table to patient registration.

19. The robotic drive system of claim 2, wherein the plurality of discrete registrations further includes a tool to table registration and a table to patient registration.

20. The robotic drive system of claim 2, wherein the combined registration further includes at least a position and orientation of at least one operating room structure.

21. The robotic drive system of claim 20, wherein the position and orientation of the at least one operating room structure is determined using a method selected from a group consisting of a direct measurement, a sensor, and a measurement using at least one image.

22. The robotic drive system of claim 2, wherein the combined registration further includes at least a position and orientation of at least the robotic drive system.

23. The robotic drive system of claim 2, wherein the combined registration further includes registration of a tool to at least one real time image of the 3D model.

24. The robotic drive system of claim 23, wherein the combined registration further includes pre-procedure image registration to the at least one real time image of the 3D model.

25. The robotic drive system of claim 23, wherein the tool to at least one real time image of the 3D model registration is registered one of manually, semi-automatically, and automatically.

26. The robotic drive system of claim 1, wherein the controller is further configured to update the registration to account for motion of the 3D model.

27. The robotic drive system of claim 1, wherein the controller is further configured to update the registration to account for motion of at least one operating room structure.

50. The robotic system of claim 1, further comprising a second sensor coupled to the tool, wherein the second sensor is configured to sense a structure of a patient'"'"'s anatomy.

51. The robotic system of claim 50, wherein the controller is configured to combine the sensed structure of the patient'"'"'s anatomy with the registration of the sensor reference frame to produce a combined registration between the sensor reference frame and the 3D model reference frame.

28. A system for controlling a medical device within a blood vessel in a patient for performing a procedure in the blood vessel, where the medical device comprises a shape sensor coupled thereto, the system comprising:

a user input device;

an actuator configured to manipulate and reposition the medical device with respect to a 3D model of the blood vessel, wherein the shape sensor on the medical device senses a shape of the medical device while in the blood vessel, and wherein the sensed shape of the medical device in the blood vessel is correlated against features of the blood vessel of the 3D model; and

a controller configured to convert a user input into a plurality of signals based on registration of a shape sensor reference frame to a patient reference frame.

29. The system of claim 28, wherein the patient reference frame is a reference frame of at least one image of an imaging device.

30. The system of claim 28, wherein the registration of the shape sensor reference frame to the patient reference frame further comprises registration of the shape sensor reference frame to a medical device reference frame and registration of the medical device reference frame to the patient reference frame.

31. The system of claim 28, wherein a pre-procedure registration is performed, the pre-procedure registration comprising registration of an imaging device with at least one image of the imaging device and registration of at least one medical appliance.

32. The system of claim 28, wherein registration of the shape sensor reference frame to the patient reference frame further comprises a shape sensor to image registration, comprising registration of the shape sensor reference frame to at least one real time image from an imaging device.

33. The system of claim 31, wherein the controller is further configured to update the pre-procedure registration by accounting for movement of the at least one medical appliance.

34. The system of claim 28, wherein the shape sensor is registered to the medical device and registration of the shape sensor to the medical device is specific to the medical device.

35. The system of claim 34, wherein the medical device includes at least one optical fiber coupled thereto, and wherein the shape sensor to medical device registration is fixed.

36. The system of claim 31, wherein the pre-procedure registration includes a medical device to actuator registration.

37. The system of claim 30, wherein the patient reference frame comprises reference frame in a fluoroscopic model of the patient comprising a plurality of fluoroscopic images.

38. The system of claim 28, wherein a plurality of discrete registrations are performed.

39. The system of claim 38, wherein the plurality of discrete registrations include an actuator to fluoroscopic imaging device registration and a fluoroscopic imaging device to fluoroscopic image registration.

40. The system of claim 28, wherein the patient reference frame comprises a reference frame in a virtual 3D model of the patient.

41. The system of claim 28, wherein the patient reference frame comprises a reference frame relative to a body of the patient in real time.

42. The system of claim 39, wherein a shape sensor to image registration includes an actuator to table registration and a table to patient registration.

43. The system of claim 42 wherein the shape sensor to image registration includes at least a position and orientation of at least one operating room structure.

44. The system of claim 43, wherein the position and orientation of the at least one operating room structure is determined using a method selected from a group consisting of a direct measurement, a sensor, and a measurement using at least one image.

45. The system of claim 32, wherein the shape sensor to image registration further comprises registration of a medical device to at least one real time image of the patient.

46. The system of claim 45, wherein the shape sensor to image registration further comprises a pre-procedure image registration to the at least one real time image of the patient.

47. The system of claim 45, wherein the medical device to at least one real time image of the patient registration is registered one of manually, semi-automated, and automatically.

48. The system of claim 32, wherein the controller is further configured to update the shape sensor to image registration to account for motion of the patient.

49. The system of claim 28, wherein the controller is further configured to update the shape sensor to the patient reference frame registration to account for motion of at least one operating room structure.