SYSTEMS AND METHODS FOR LOCALIZING, TRACKING AND/OR CONTROLLING MEDICAL INSTRUMENTS

• US 20140275985A1
• Filed: 03/15/2013
• Published: 09/18/2014
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Auris Health, Inc. (Johnson & Johnson)
• Original Assignee: Hansen Medical Incorporated (Johnson & Johnson)
• Inventors: Walker, Sean, Sewell, Christopher, Park, June, Koolwal, Aditya, Camarillo, Dave, Barbagli, Frederico, Ravindran, Prabu
• Granted Patent: US 9,629,595 B2
• Time in Patent Office: Days
• US Class Current: 600/424
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/301 for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes
• CPC Class Codes: A61B 2090/376 using X-rays, e.g. fluoroscopy
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 5/062 using magnetic field
• CPC Class Codes: A61B 5/066 Superposing sensor position on an image of the patient, e.g. obtained by ultrasound or x-ray imaging
• CPC Class Codes: A61B 5/725 using specific filters therefor, e.g. Kalman or adaptive filters A61B5/316 take precedence
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/12 Arrangements for detecting or locating foreign bodies
• CPC Class Codes: A61B 6/485 involving fluorescence X-ray imaging
• CPC Class Codes: A61B 6/5205 involving processing of raw data to produce diagnostic data
• CPC Class Codes: A61B 6/5258 involving detection or reduction of artifacts or noise

Abstract

Systems and methods are described herein for tracking, localization or controlling an elongate instrument or other medical instrument in an image or patient.

Claims

1. A method of tracking an elongate instrument in real time in an image comprising:

initializing an active contour in the image wherein the active contour corresponds to at least a portion of the elongate instrument; and

updating the active contour as the elongate instrument moves by performing a search based on pixel intensity to track the elongate instrument.

2. The method of claim 1, further comprising performing an image-based search along the active contour to track one or more features of the elongate instrument.

3. The method of claim 2, wherein the image-based search along the active contour is performed with a template matching search.

4. The method of claim 1, wherein an image-based search or template matching search is performed to track a feature, marker or point of interest of the elongate instrument in order to enhance or confirm accuracy of the active contour.

5. The method of claim 1, wherein the active contour restricts the area in which an image-based search or template matching search must occur and provides a seed for the orientation of a template.

6. The method of claim 1, wherein once an active contour is tracking the elongate instrument, an image-based search or template matching search is performed to track a specific feature, marker or point of interest of the elongate instrument to ascertain a location of the specific feature, marker or point of interest on the active contour.

7. The method of claim 6, wherein a feature or point of interest includes an articulation section, control ring, marker, or a sheath of the elongate instrument.

8. The method of claim 1, wherein the tracking and/or an image-based search or template matching search facilitates instinctive driving of the elongate instrument.

9. The method of claim 1, wherein an image-based search or template matching search is performed to track a heading direction of an articulation section of the elongate instrument to facilitates instinctive driving of the elongate instrument.

10. The method of claim 1, wherein the image is a fluoroscopy image.

11. The method of claim 1, further comprising maintaining spacing between at least two points in a sequence of points along the active contour to prevent the active contour from mis-tracking in areas where the elongate instrument curves rapidly.

12. The method of claim 1, further comprising filtering out high frequency texture and noise in the image with a low-pass spatial filter.

13. The method of claim 1, further comprising filtering out low frequency features such as anatomy, imaging artifacts, or elements in the environment with a high-pass spatial filter.

14. The method of claim 1, further comprising performing a correlation matching search along the active contour to track features of the elongate instrument and to enhance or confirm accuracy of the active contour.

15. The method of claim 1, wherein an image-based search or template matching search is a low computation search using pixel sub sampling.

16. The method of claim 1, wherein the active contour is updated by using an iterative approach.

17. The method of claim 1, wherein the active contour can grow and shrink as a projection of the elongate instrument on the image grows and shrinks.

18. The method of claim 1, wherein each point along the active contour produces a fitness value that can guide whether or not a point is on the elongate instrument in the image.

19. The method of claim 1, wherein points are added or removed at a desired distance from an end point of the active contour in the direction that the active contour is traveling to grow or shrink the active contour to track movement of the elongate instrument.

20. The method of claim 1, wherein an active contour may be initialized by selecting two or more points along a shape or path of the elongate instrument in the image using a pointing or selection mechanism.

21. The method of claim 1, wherein an active contour may be initialized by identifying points on a leader or control ring marker and/or a sheath marker of the elongate instrument, wherein the points initialize the active contour and provide a starting location for tracking the leader or control ring marker and/or the sheath marker.

22. The method of claim 1, wherein a template matching search is performed and templates vary based on the type and size of a feature on the elongate instrument.

23. The method of claim 22, wherein a particle filter is used to guide template matching.

24. The method of claim 1, further comprising maintaining a second contour for tracking other elongate objects and artificially removing the objects from the image.

25. The method of claim 1, wherein one or more computer vision algorithms track elements in the image such as surgical tools, other elongate instruments, or anatomical features in order to aid the active contour in overcoming those elements overlapping and obscuring the elongate instrument being tracked.

26. The method of claim 1, further comprising providing a tracking view overlaid on a fluoroscopy image wherein a user'"'"'s view of the fluoroscopy image may be augmented.

27. The method of claim 1, further comprising predicting elongate instrument or active contour movement based on a known commanded elongate instrument motion to increase tracking accuracy.

28. The method of claim 1, wherein the elongate instrument is a robotically controlled elongate instrument.

29. The method of claim 1, wherein the image comprises multiple fluoroscopy views from different angles acquired using one or more fluoroscopy systems.

30. A method of tracking a robotically controlled elongate instrument in real time in an image comprising:

controlling movement of the elongate instrument with a robotic or system command;

creating an active contour which corresponds to at least a portion of the elongate instrument;

updating the active contour as the elongate instrument moves;

performing an image-based or template matching search along the active contour to track features of the elongate instrument; and

predicting elongate instrument movement based on the commanded elongate instrument motion to increase tracking accuracy.

31. The method of claim 30, wherein a known command for elongate instrument motion is utilized to predict a location, movement or shape of the active contour in the image.

32. The method of claim 30, wherein a known command for elongate instrument motion is utilized to focus a pixel intensity search to update the active contour or to focus the image-based or template matching search to track features of the elongate instrument in a specific portion of the image.

33. The method of claim 30, wherein a known command for elongate instrument motion is utilized to eliminate false positives during tracking.

34. The method of claim 30, wherein a known distance between a first point and a second point on the elongate instrument is used to guide or focus the image-based or template matching search for the first point based on the position of the second point.

35. The method of claim 30, wherein a known command for elongate instrument motion is utilized to give guidance on the scale of pixels per millimeter on an image.

36. The method of claim 30, wherein a known command for elongate instrument motion is utilized to give guidance on the amount or distance that an elongate instrument is in or out of plane or moving in or out of plane.

37. The method of claim 30, wherein a known distance between a first point and a second point on the elongate instrument is used to determine the scale of the active contour in the image.

38. The method of claim 30, wherein the image is a fluoroscopy image.

39. The method of claim 30, wherein the image-based search along the active contour is performed with a template matching search.

40. A robotic system for controlling an elongate instrument with respect to a target space comprising:

a robotic drive system comprising at least one actuator, the robotic drive system configured to interchangeably couple with the elongate instrument to position the instrument with respect to the target space;

a controller configured to initialize an active contour in an image, wherein the active contour corresponds to at least a portion of the elongate instrument, and wherein the controller is configured to update the active contour as the elongate instrument moves by performing a search based on pixel intensity, such that the controller can produce a plurality of signals to direct the robotic drive system or elongate instrument in the image frame based on tracking of the elongate instrument with the active contour.

41. The system of claim 40, wherein the controller is configured to perform an image-based search along the active contour to track one or more features of the elongate instrument.

42. The system of claim 41, wherein the image-based search along the active contour is performed with a template matching search.

43. The system of claim 40, wherein the controller is configured to perform and image-based search or template matching search to track a feature, marker or point of interest of the elongate instrument in order to enhance or confirm accuracy of the active contour.

44. The system of claim 40, wherein the active contour restricts the area in which an image-based search or template matching search must occur and provides a seed for the orientation of a template.

45. The system of claim 40, wherein the controller is configured to perform an image-based search or template matching search to track a specific feature, marker or point of interest of the elongate instrument to ascertain a location of the specific feature, marker or point of interest on the active contour.

46. The system of claim 45, wherein a feature or point of interest includes an articulation section, control ring, marker, or a sheath of the elongate instrument.

47. The system of claim 40, wherein the controller is configured to perform tracking and/or an image-based search or template matching search to facilitate instinctive driving of the elongate instrument.

48. The system of claim 40, wherein the controller is configured to perform an image-based search or template matching search to track a heading direction of an articulation section of the elongate instrument to facilitate instinctive driving of the elongate instrument.

49. The system of claim 40, wherein the image is a fluoroscopy image.

50. The system of claim 40, further comprising a low-pass spatial filter for filtering high frequency texture and noise in the image.

51. The system of claim 40, further comprising a master input device coupled to the controller, where the master input device uses the active contour or image based search tracking to permit intuitive or instinctive driving of the tool using the robotic drive system.

52. The system of claim 40, wherein the image comprises an image of the target space or patient.

53. A method of controlling a robotically controlled elongate instrument in real time comprising:

displaying an image of an anatomy of a patient;

tracking or detecting a localization sensor coupled to the robotically controlled elongate instrument; and

registering localization data from the localization sensor to the image to provide a continuously updated location of at least a portion of the elongate instrument in the image of the anatomy of a patient to facilitate robotic navigation of the elongate instrument through the anatomy.

54. The method of claim 53, wherein the localization sensor is an electromagnetic localization sensor.

55. The method of claim 54, wherein the electromagnetic localization sensor is configured to be placed in a pulsating magnetic field generated by an electromagnetic field generator to allow for sensor detection or tracking.

56. The method of claim 53, wherein a position and/or orientation of the localization sensor is continuously tracked to allow for accurate manipulation of the elongate instrument.

57. The method of claim 53, wherein the image is generated by CT or 2D or 3D fluoroscopy.

58. The method of claim 53, wherein registering comprises transforming localization data generated by the localization sensor to the coordinate frame of the image such that localization data of the elongate instrument is overlaid on the image.

59. The method of claim 53, wherein the image is a 3D or 2D anatomical model or a 2D or 3D fluoroscopic image.

60. The method of claim 53, further comprising providing an image intensifier, wherein localization data from the localization sensor is registered to a fluoroscopy coordinate system of a fluoroscopy image coupled to an image intensifier.

61. The method of claim 60, wherein order to register the localization data from the localization sensor to a fluoroscopy image, one or more of the following parameters are ascertained or known:

a distance from an X-ray source to the image intensifier, a distance from the source to a bed, a size of the image intensifier, and the axis of rotation of a c-arm of the fluoroscopy system.

62. The method of claim 53, wherein the robotically controlled elongate instrument is a vascular catheter.

63. The method of claim 53, wherein the registration is used to facilitate intuitive or instinctive driving of the elongate instrument.

64. A system for tracking or localizing a robotically controlled elongate instrument comprising:

an image of an anatomy of a patient;

an electromagnetic localization sensor coupled to an elongate instrument; and

an electromagnetic field generator, wherein the generator is configured to produce an electromagnetic field in which the electromagnetic localization sensor is detected, and wherein the localization sensor is configured to provide localization data for at least a portion of the elongate instrument, wherein the localization data is registered to the image to provide a continuously updated location of at least a portion of the elongate instrument in the image to facilitate robotic navigation of the elongate instrument through the anatomy.

65. The system of claim 64, further comprising a controller configured to register the localization data to the image to provide a continuously updated location of at least a portion of the elongate instrument in the image such that the controller can produce a plurality of signals to direct robotic navigation of the elongate instrument through the anatomy based on the location of the at least a portion of the elongate instrument in the image.

66. The system of claim 64, wherein the system is configured such that a position and/or orientation of the localization sensor is measureable in a coordinate system of the electromagnetic field generator, and wherein the position and/or orientation measurement of the localization sensor is registered to an image via one or more transformations.

67. The system of claim 64, wherein the image is a 3D or 2D anatomical model or 2D or 3D fluoroscopic image.

68. The system of claim 64, further comprising an image intensifier, wherein localization data from the localization sensor is registered to a fluoroscopy coordinate system of a fluoroscopy image coupled to the image intensifier.

69. The system of claim 68, wherein order to register the localization data from the localization sensor to a fluoroscopy image, one or more of the following parameters are ascertained or known:

a distance from an X-ray source to the image intensifier, a distance from an X-ray source to a bed, a size of the image intensifier, and the axis of rotation of a c-arm of a fluoroscopy system.

70. A system for tracking or localizing a robotically controlled elongate instrument comprising:

an image of an anatomy of a patient;

an electromagnetic localization sensor coupled to an elongate instrument;

an electromagnetic field generator; and

at least one fixed reference sensor positioned in a workspace of the electromagnetic field generator, wherein the electromagnetic field generator is movable relative to the reference sensor thereby expanding the workspace for elongate instrument tracking.

71. The system of claim 70, wherein the reference sensor provides a reference coordinate system that is independent of a placement of the electromagnetic field generator.

72. The system of claim 71, wherein the localization sensor is configured to provide localization data for at least a portion of the elongate instrument, wherein the localization data is registered to the image via the reference coordinate system to provide a continuously updated location of at least a portion of the elongate instrument in the image to facilitate robotic navigation of the elongate instrument through the anatomy.

73. The system of claim 72, wherein the electromagnetic field generator is configured to be moved without distorting a measurement of the localization sensor.

74. The system of claim 71, wherein the system is configured such that a location of the localization sensor is measurable in the reference sensor coordinate system, wherein the location measurement of the localization sensor is registered to the image via one or more transformations.

75. The system of claim 74, wherein a transformation is fixed in space and does not require online computation.

76. The system of claim 70, wherein the reference sensor comprises an anatomy patch, wherein the anatomy patch is configured to be positioned on a patient such that the reference sensor appears in the image and a position and/or orientation of the reference sensor can be measured in the coordinate system of the image.

77. The system of claim 70, wherein the image is 3D or 2D anatomical model or a 2D or 3D fluoroscopic image.

78. The system of claim 70, further comprising an image intensifier, wherein localization data from the localization sensor is registered to a fluoroscopy coordinate system of a fluoroscopy image coupled to the image intensifier.

79. The system of claim 78, wherein order to register the localization data from the localization sensor to a fluoroscopy image, one or more of the following parameters are ascertained or known:

a distance from an X-ray source to the image intensifier, a distance from an X-ray source to a bed, a size of the image intensifier, and the axis of rotation of a c-arm of the fluoroscopy machine.

80. A method of tracking or localizing a robotically controlled elongate instrument in real time comprising:

displaying a single image of an anatomy of a patient;

tracking or detecting a localization sensor coupled to the robotically controlled elongate instrument; and

registering localization data from the localization sensor to the image via a reference sensor to provide a continuously updated location of at least a portion of the elongate instrument in the image of the anatomy of a patient to facilitate robotic navigation of the elongate instrument through the anatomy.

81. The method of claim 80, wherein only a single image is required to register the localization data.

82. The method of claim 80, wherein a position and/or orientation measurement of the localization sensor is registered to the image via one or more transformations.

83. The method of claim 80, wherein a fluoroscope for creating the displayed image remains fixed and is not required to rotate or move to complete the registration.

84. The method of claim 80, wherein a bed sensor and a fixed reference sensor are utilized in performing the registration, and a frame transformation between the bed and the reference sensor is performed.

85. The method of claim 80, wherein the registration is used to facilitate intuitive or instinctive driving of the elongate instrument.

86. A medical system for controlling an elongate instrument comprising:

a robotically controlled instrument assembly comprising a shapeable elongate instrument;

a localization system coupled to the robotically controlled instrument assembly and configured to track the shapeable elongate instrument, wherein at least a portion of the localization system is sterilely isolated from the robotically controlled instrument assembly.

87. The system of claim 86, wherein the localization system comprises a localization sensor coupled to the shapeable elongate instrument within a sterile field and the localization sensor is coupled to a sensor detection unit positioned outside of the sterile field via a connector positioned within the sterile field.

88. The system of claim 86, wherein the localization system comprises a localization sensor coupled to the shapeable elongate instrument within a sterile field and the localization sensor is coupled to a sensor detection unit positioned outside of the sterile field via a wire and a connector, wherein at least a portion of the wire is positioned in the sterile field and the connector is positioned outside of the sterile field.

89. The system of claim 88, further comprising a second connector for coupling the localization sensor to the sensor detection unit, wherein the second connector is positioned within the sterile field and wherein the second connector is resistant to bodily fluids

90. The system of claim 86, wherein the localization system comprises a localization sensor coupled to the shapeable elongate instrument within a sterile field and the localization sensor is coupled to a sensor detection unit positioned outside of the sterile field via a wire and a connector, wherein the connector is implemented under the robotically controlled instrument assembly and the entire wire is positioned outside of the sterile field.

91. The system of claim 86, wherein the localization system comprises a localization sensor coupled to the shapeable elongate instrument within a sterile field and the localization sensor is coupled to a sensor detection unit, wherein the sensor detection unit is integrated in an instrument driver coupled to the robotically controlled instrument assembly, wherein the sensor detection unit is configured to communicate via a digital signal with a sensor controller positioned outside of the sterile field.

92. The system of claim 86, wherein the localization system comprises a localization sensor coupled to the shapeable elongate instrument within a sterile field and the localization sensor is coupled to a sensor detection unit, wherein the sensor detection unit is integrated in the robotically controlled instrument assembly, wherein the sensor detection unit is configured to wirelessly communicate with a sensor controller positioned outside of the sterile field.

93. The system of claim 92, wherein the robotically controlled instrument assembly is disposable.

94. The system of claim 86, wherein the localization system comprises an electromagnetic localization sensor.

95. A robotic system for controlling an elongate instrument with respect to a target space comprising:

an elongate instrument having a localization sensor coupled thereto;

a robotic drive system comprising at least one actuator, the robotic drive system configured to interchangeably couple with the elongate instrument to position the instrument with respect to the target space;

a controller configured to produce a registration between a localization sensor frame and an image frame or a patient frame, such that the controller can produce a plurality of signals to direct the robotic drive system or elongate instrument in the image frame using the registration and wherein the image comprises an image of the target space.

96. The system of claim 95, wherein the localization sensor is an electromagnetic localization sensor.

97. The system of claim 96, wherein the electromagnetic localization sensor is configured to be placed in a pulsating magnetic field generated by an electromagnetic field generator to allow for sensor detection or tracking.

98. The system of claim 95, wherein a position and/or orientation of the localization sensor is continuously tracked to allow for accurate manipulation of the elongate instrument.

99. The system of claim 95, wherein the image is generated by CT or 2D or 3D fluoroscopy.

100. The system of claim 95, wherein registering comprises transforming localization data generated by the localization sensor to the coordinate frame of the image such that localization data of the elongate instrument is overlaid on the image.

101. The system of claim 95, wherein the image is a 3D or 2D anatomical model or a 2D or 3D fluoroscopic image.

102. The system of claim 95, further comprising providing an image intensifier, wherein localization data from the localization sensor is registered to a fluoroscopy coordinate system of a fluoroscopy image coupled to an image intensifier.

103. The system of claim 102, wherein registration to a fluoroscopy image is produced by knowing:

a distance from an X-ray source to the image intensifier, a distance from the source to a bed, a size of the image intensifier, and/or the axis of rotation of a c-arm of a fluoroscopy system.

104. The system of claim 95, wherein the elongate instrument is a robotically controlled vascular catheter.

105. The system of claim 95, wherein a single image or multiple images are utilized.

106. The system of claim 95, further comprising a master input device coupled to the controller, where the master input device uses the registration to permit intuitive or instinctive driving of the elongate instrument using the robotic drive system.

107. A robotic system for controlling an elongate instrument with respect to a target space comprising:

an elongate instrument having a localization sensor coupled thereto;

a robotic drive system comprising at least one actuator, the robotic drive system configured to interchangeably couple with the elongate instrument to position the instrument with respect to the target space; and

a controller configured to register localization data from the localization sensor to an image of an anatomy or to a patient or target space frame to provide a continuously updated location of at least a portion of the elongate instrument in the image, such that the controller can produce a plurality of signals to direct robotic navigation of the elongate instrument through the anatomy based on the location of the at least a portion of the elongate instrument in the image.

108. The system of claim 107, wherein the controller is configured to convert a user input into a plurality of signals based on the registration of a sensor reference frame to the image frame or target space or patient reference frame.