SYSTEM AND METHOD FOR MEDICAL INSTRUMENT NAVIGATION AND TARGETING

• US 20190175062A1
• Filed: 12/06/2018
• Published: 06/13/2019
• Est. Priority Date: 12/08/2017
• Status: Active Grant

• Current Assignee: Auris Health, Inc. (Johnson & Johnson)
• Original Assignee: Auris Health, Inc. (Johnson & Johnson)
• Inventors: Rafii-Tari, Hedyeh, Jeevan, Prasanth, Boyd, Douglas T., Teran, Melissa A., Sheehy, Alexander James, Robert, Nicolas E., Drahos, Miroslav, Howard, Jeffery D., Zeldis, Andrew Esbenshade, Mambembe, Rene Ango
• Granted Patent: US 10,835,153 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 2017/00809 Lung operations
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2059 Mechanical position encoders
• CPC Class Codes: A61B 2034/2061 using shape-sensors, e.g. fiber shape sensors with Bragg gratings
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2034/256 having a database of accessory information, e.g. including context sensitive help or scientific articles
• CPC Class Codes: A61B 2034/258 providing specific settings for specific users
• CPC Class Codes: A61B 2034/301 for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes
• CPC Class Codes: A61B 2090/306 using optical fibres
• CPC Class Codes: A61B 2090/309 using white LEDs
• CPC Class Codes: A61B 2090/3614 using optical fibre
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/368 changing the image on a display according to the operator's position
• CPC Class Codes: A61B 2090/372 Details of monitor hardware
• CPC Class Codes: A61B 2090/376 using X-rays, e.g. fluoroscopy
• CPC Class Codes: A61B 2090/3762 using computed tomography systems [CT]
• CPC Class Codes: A61B 2090/502 Headgear, e.g. helmet, spectacles
• CPC Class Codes: A61B 2217/005 with suction drainage system
• CPC Class Codes: A61B 2217/007 : with irrigation system
• CPC Class Codes: A61B 34/20 : Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/25 : User interfaces for surgical systems
• CPC Class Codes: A61B 34/30 : Surgical robots
• CPC Class Codes: A61B 34/74 : Manipulators with manual electric input means
• CPC Class Codes: A61B 5/066 : Superposing sensor position on an image of the patient, e.g. obtained by ultrasound or x-ray imaging
• CPC Class Codes: A61B 90/37 : Surgical systems with images on a monitor during operation

Abstract

Certain aspects relate to systems and techniques for medical instrument navigation and targeting. In one aspect, a system includes a medical instrument having an elongate body and at least one sensor, a display, a processor, and a memory storing a model of a mapped portion of a luminal network and a position of a target with respect to the model. The processor may be configured to: determine, based on data from the at least one sensor, a position and orientation of a distal end of the medical instrument with respect to the model, and cause, on at least a portion of the display, a rendering of the model, the position of the target, and the position and orientation of the distal end of the medical instrument. The rendering may be based on a viewpoint directed at the target and different from a viewpoint of the medical instrument

Claims

1. A system, comprising:

a medical instrument having an elongate body and at least one sensor;

a display;

a processor; and

a memory storing a model of a mapped portion of a luminal network and a position of a target with respect to the model, the memory further storing computer-executable instructions to cause the processor to;

determine, based on data from the at least one sensor, a position and orientation of a distal end of the medical instrument with respect to the model, and

cause, on at least a portion of the display, a rendering of the model, the position of the target, and the position and orientation of the distal end of the medical instrument, wherein the rendering is based on a viewpoint directed at the target and different from a viewpoint of the medical instrument.

2. The system of claim 1, further comprising a user input element, wherein the memory further stores computer-executable instructions to cause the processor to:

receive, via the user input element, a first type of user interaction with the user input element,

generate a first type of rendering instruction based on the first type of user interaction with the user input element, and

in response to generating the first type of rendering instruction, cause within the portion of the display a rendering of a modified view of the model, the position of the target, and the position and orientation of the distal end of the medical instrument.

3. The system of claim 2, wherein:

the first type of rendering instruction comprises a rotation instruction to rotate the viewpoint directed at the target, and

the modified view comprises a rotated view of the model, the position of the target, and the position and orientation of the distal end of the medical instrument, with the target as a center of the rotation.

4. The system of claim 2, wherein the memory further stores computer-executable instructions to cause the processor to:

receive, via the user input element, a second type of user interaction with the user input element,

generate a second type of rendering instruction based on the second type of user interaction with the user input element, the second type of rendering instruction comprising an elevation instruction to alter an angle between a reference plane centered at the target and the viewpoint directed at the target, and

in response to generating the second type of rendering instruction, cause within the portion of the display a rendering of the of the model having an altered the angle between the reference plane and the viewpoint directed at the target.

5. The system of claim 4, wherein:

the user input element comprises a button,

the first user interaction type comprises an actuation of the button for a greater than a threshold period of time, and

the second user interaction type comprises an actuation of the button for less than the threshold period of time.

6. The system of claim 2, wherein:

the first type of rendering instruction comprising a toggle perspective instruction to toggle the viewpoint of the rendering from the viewpoint different from the viewpoint of the medical instrument to the viewpoint of the medical instrument, and

the modified view comprises a rendering of the model from the viewpoint of the medical instrument.

7. The system of claim 1, wherein the memory further stores computer-executable instructions to cause the processor to:

cause, based on the position and orientation of the distal end of the medical instrument, a rendering of a graphical indicator extending from the distal end of the medical instrument in an insertion direction of the medical instrument,

determine, based on the data from the at least one sensor, a change in at least one of the position and orientation of the distal end of the medical instrument,

determine, in response to determining the change in at least one of the position and orientation of the distal end of the medical instrument, that the graphical indicator intersects the target, and

cause a rendering of a change to the graphical indicator in response to determining that the line intersects the target.

8. The system of claim 1, wherein the memory further stores computer-executable instructions to cause the processor to:

determine, based on the data from the at least one sensor, that the distal end of the medical instrument is within a threshold distance from the target; and

cause a rendering of a graphical indicator in response to determining that the distal end of the medical instrument is within a threshold distance from the target, the graphical indicator extending from the distal end of the medical instrument in an insertion direction of the medical instrument.

9. The system of claim 8, wherein the graphical indicator is selected from:

a line, a cylinder, or a cone.

10. The system of claim 8, wherein the graphical indicator is a cone, the aperture of the cone is based on an estimated error range of the orientation of the distal end of the medical instrument.

11. The system of claim 8, wherein the memory further stores computer-executable instructions to cause the processor to:

determine, in response to determining the change in at least one of the position and orientation of the distal end of the medical instrument, that the graphical indicator intersects a center of the target, and

cause a rendering of a change in the color of the graphical indicator in response to determining that the graphical indicator intersects the center of the target.

12. The system of claim 1, wherein:

the model comprises a skeleton comprising a plurality of segments, wherein each of the segments is defined with respect to a center line of a corresponding lumen of the mapped portion of the luminal network,

the memory further stores computer-executable instructions to cause the processor to;

identify a segment in which the distal end of the medical instrument is located,

determine a depth of the distal end of the medical instrument along the identified segment, and

determine the position of the distal end of the medical instrument based on the identified segment and the depth.

13. The system of claim 12, wherein the memory further stores computer-executable instructions to cause the processor to:

determine a distance between the distal end of the medical instrument and the target based on the determined position of the distal end of the medical instrument and the position of the target.

14. The system of claim 12, wherein the memory further stores computer-executable instructions to cause the processor to:

cause, on at least another portion of the display, a rendering of the position of the distal end of the medical instrument based on the determined position of the distal end of the medical instrument.

15. The system of claim 12, wherein the memory further stores computer-executable instructions to cause the processor to:

determine that the position of the distal end of the medical instrument is within a volume defined by the model, and

restrict the determined position of the distal end of the medical instrument to be located along the skeleton.

16. The system of claim 12, wherein:

the at least one sensor comprises one of;

an electromagnetic (EM) sensor or a shape sensing fiber, and

the memory further stores computer-executable instructions to cause the processor to configured generate the position data based only on data received from one of the EM sensor of the shape sensing fiber.

17. A non-transitory computer readable storage medium having stored thereon instructions that, when executed, cause at least one computing device to:

determine, based on data from at least one sensor of a medical instrument, a position and orientation of a distal end of the medical instrument with respect to a model of a mapped portion of a luminal network, and

cause, on at least a portion of a display, a rendering of the model, a position of a target with respect to the model, and the position and orientation of the distal end of the medical instrument, wherein the rendering is based on a viewpoint directed at the target and different from a viewpoint of the medical instrument.

18. The non-transitory computer readable storage medium of claim 17, further having stored thereon instructions that, when executed, cause at least one computing device to:

receive, via the user input element, a first type of user interaction with the user input element;

generate a first type of rendering instruction based on the first type of user interaction with the user input element; and

in response to generating the first type of rendering instruction, cause within the portion of the display a rendering of a modified view of the model, the position of the target, and the position and orientation of the distal end of the medical instrument.

19. The non-transitory computer readable storage medium of claim 18, wherein:

the first type of rendering instruction comprises a rotation instruction to rotate the viewpoint directed at the target, and

the modified view comprises a rotated view of the model, the position of the target, and the position and orientation of the distal end of the medical instrument, with the target as a center of the rotation.

20. The non-transitory computer readable storage medium of claim 18, further having stored thereon instructions that, when executed, cause at least one computing device to:

receive, via the user input element, a second type of user interaction with the user input element;

generate a second type of rendering instruction based on the second type of user interaction with the user input element, the second type of rendering instruction comprising an elevation instruction to alter an angle between a reference plane centered at the target and the viewpoint directed at the target; and

in response to generating the second type of rendering instruction, cause within the portion of the display a rendering of the of the model having an altered the angle between the reference plane and the viewpoint directed at the target.

21. The non-transitory computer readable storage medium of claim 20, wherein:

the user input element comprises a button,

the first user interaction type comprises an actuation of the button for a greater than a threshold period of time, and

the second user interaction type comprises an actuation of the button for less than the threshold period of time.

22. The non-transitory computer readable storage medium of claim 18, wherein:

the first type of rendering instruction comprising a toggle perspective instruction to toggle the viewpoint of the rendering from the viewpoint different from the viewpoint of the medical instrument to the viewpoint of the medical instrument, and

the modified view comprises a rendering of the of the model from the viewpoint of the medical instrument.

23. The non-transitory computer readable storage medium of claim 17, further having stored thereon instructions that, when executed, cause at least one computing device to:

cause, based on the position and orientation of the distal end of the medical instrument, a rendering of a graphical indicator extending from the distal end of the medical instrument in an insertion direction of the medical instrument;

determine, based on the data from the at least one sensor, a change in at least one of the position and orientation of the distal end of the medical instrument;

determine, in response to determining the change in at least one of the position and orientation of the distal end of the medical instrument, that the graphical indicator intersects the target; and

cause a rendering of a change to the graphical indicator in response to determining that the line intersects the target.

24. The non-transitory computer readable storage medium of claim 17, further having stored thereon instructions that, when executed, cause at least one computing device to:

determine, based on the data from the at least one sensor, that the distal end of the medical instrument is within a threshold distance from the target; and

cause a rendering of a graphical indicator in response to determining that the distal end of the medical instrument is within a threshold distance from the target, the graphical indicator extending from the distal end of the medical instrument in an insertion direction of the medical instrument.

25. The non-transitory computer readable storage medium of claim 24, further having stored thereon instructions that, when executed, cause at least one computing device to:

determine, in response to determining the change in at least one of the position and orientation of the distal end of the medical instrument, that the graphical indicator intersects a center of the target, and

cause a rendering of a change in the color of the graphical indicator in response to determining that the graphical indicator intersects the center of the target.

26. The non-transitory computer readable storage medium of claim 25, wherein:

the model comprises a skeleton comprising a plurality of segments, wherein each of the segments is defined with respect to a center line of a corresponding lumen of the mapped portion of the luminal network,

the non-transitory computer readable storage medium further has stored thereon instructions that, when executed, cause at least one computing device to;

identify a segment in which the distal end of the medical instrument is located;

determine a depth of the distal end of the medical instrument along the identified segment; and

determine the position of the distal end of the medical instrument based on the identified segment and the depth.

27. The non-transitory computer readable storage medium of claim 26, further having stored thereon instructions that, when executed, cause at least one computing device to:

determine a distance between the distal end of the medical instrument and the target based on the determined position of the distal end of the medical instrument and the position of the target.

28. The non-transitory computer readable storage medium of claim 26, further having stored thereon instructions that, when executed, cause at least one computing device to:

cause, on at least another portion of the display, a rendering of the position of the distal end of the medical instrument based on the determined position of the distal end of the medical instrument.

29. The non-transitory computer readable storage medium of claim 26, further having stored thereon instructions that, when executed, cause at least one computing device to:

determine that the position of the distal end of the medical instrument is within a volume defined by the model, and

restrict the determined position of the distal end of the medical instrument to be located along the skeleton.

30. The non-transitory computer readable storage medium of claim 26, wherein:

the at least one sensor comprises one of;

an electromagnetic (EM) sensor or a shape sensing fiber, and

the memory further stores computer-executable instructions to cause the processor to configured generate the position data based only on data received from one of the EM sensor of the shape sensing fiber.