ROBOTIC SYSTEM CONFIGURED FOR NAVIGATION PATH TRACING

• US 20190110839A1
• Filed: 09/26/2018
• Published: 04/18/2019
• Est. Priority Date: 10/13/2017
• Status: Active Grant

• Current Assignee: Auris Health, Inc. (Johnson & Johnson)
• Original Assignee: Auris Health, Inc. (Johnson & Johnson)
• Inventors: Rafii-Tari, Hedyeh, Jeevan, Prasanth
• Granted Patent: US 11,058,493 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 2017/00477 Coupling A61B2017/0046 takes precedence
• CPC Class Codes: A61B 2017/00809 Lung operations
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2059 Mechanical position encoders
• CPC Class Codes: A61B 2034/2061 using shape-sensors, e.g. fiber shape sensors with Bragg gratings
• CPC Class Codes: A61B 2034/252 indicating steps of a surgical procedure
• CPC Class Codes: A61B 2034/301 for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes
• CPC Class Codes: A61B 2090/306 using optical fibres
• CPC Class Codes: A61B 2090/309 using white LEDs
• CPC Class Codes: A61B 2090/3614 using optical fibre
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 90/37 Surgical systems with images on a monitor during operation

Abstract

Certain aspects relate to systems and techniques for navigation path tracing. In one aspect, a system displays a preoperative model of a luminal network is displayed. The system determines a position of an instrument within the luminal network relative to the preoperative model. Based on the position of the instrument relative to the preoperative model, the system determines whether to enter a path tracing mode. In path tracing mode the system displays visual indicia indicative of a path of the instrument with respect to the displayed preoperative model. The visual indicia may be used to visual the navigation path of the instrument and/or to extend the preoperative model.

Claims

1. A non-transitory computer readable storage medium having stored thereon instructions that, when executed, cause a processor of a device to at least:

display, on a user display, a preoperative model corresponding to a mapped portion of a luminal network of a patient;

determine a position of a distal end of an instrument that is positioned within the luminal network relative to the mapped portion of the preoperative model;

based on the position of the distal end of the instrument relative to the mapped portion of the luminal network, enter a path tracing mode; and

when in the path tracing mode, display, on the user display, visual indicia indicative of a path of the distal end of the instrument with respect to the displayed preoperative model.

2. The non-transitory computer readable storage medium of claim 1, wherein the instructions are configured to cause the processor of the device to enter the path tracing mode when the position of the distal end of the instrument is outside the mapped portion of the preoperative model.

3. The non-transitory computer readable storage medium of claim 2, wherein the instructions are configured to cause the processor of the device to deactivate the path tracing mode when the position of the distal end of the instrument is inside the mapped portion of the preoperative model.

4. The non-transitory computer readable storage medium of claim 1, wherein the instructions are configured to cause the processor of the device to enter the path tracing mode when the position of the distal end of the instrument is within 25%, 20%, 15%, 10%, or 5% an end of a last segment of the preoperative model.

5. The non-transitory computer readable storage medium of claim 1, wherein the visual indicia are indicative of historical positions of the distal end of the instrument within the luminal network.

6. The non-transitory computer readable storage medium of claim 1, wherein, in in path tracing mode, the instructions are configured to cause the processor of the device to adjust a frequency of the visual indicia based on a distance traveled by the instrument between the visual indicia.

7. The non-transitory computer readable storage medium of claim 1, wherein, in in path tracing mode, the instructions are configured to cause the processor of the device to adjust a frequency of the visual indicia based on a time elapsed between the visual indicia.

8. The non-transitory computer readable storage medium of claim 1, wherein the instructions are configured to cause the processor of the device to determine the position of the distal end of the instrument based on a plurality of navigation modalities when the instrument is positioned within the mapped portion of the preoperative model, wherein the plurality of navigation modalities comprise a plurality of preoperative model data, vision data, position sensor data, and robotic command and kinematics data.

9. The non-transitory computer readable storage medium of claim 8, wherein the instructions are configured to cause the processor of the device to determine the position of the distal end of the instrument based on less than the plurality of navigation modalities when the instrument is positioned outside the mapped portion of the preoperative model.

10. The non-transitory computer readable storage medium of claim 1, wherein the instructions are configured to cause the processor of the device to associate vision data with the visual indicia, wherein the vision data comprises an image received from an imaging device on the distal end of the instrument.

11. The non-transitory computer readable storage medium of claim 1, wherein the instructions are configured to cause the processor of the device to associate robotic command and kinematics data with the visual indicia.

12. The non-transitory computer readable storage medium of claim 1, wherein the instructions are configured to cause the processor of the device to:

receive user input data from a user input; and

associate the user input data with the visual indicia;

wherein the user input data comprises one or more of;

an indication of a lumen traveled;

an indication of a lumen not traveled;

an indication of an end of a lumen;

an indication of an opening of a lumen;

an indication that a current lumen extends beyond a current position of the instrument; and

a lumen diameter.

13. A robotic system for navigating a luminal network, the system comprising:

an instrument having an elongate body and a sensor disposed on a distal end of the elongate body;

at least one computer-readable memory having stored thereon executable instructions; and

one or more processors in communication with the at least one computer-readable memory and configured to execute the instructions to cause the system to at least;

access a preoperative model of a mapped portion of the luminal network and display the preoperative model on a user display;

determine a position of the distal end of the instrument within the luminal network relative to the preoperative model using the sensor;

detect, based on the determined position relative to the preoperative model, when the distal end of the instrument has moved beyond the mapped portion of the luminal network; and

when the distal end of the instrument has moved beyond the mapped portion of the luminal network, display visual indicia of the position of the distal end of the instrument with respect to the displayed preoperative model on the user display.

14. The system of claim 13, wherein the one or more processors are configured to execute the instructions to cause the system to register a coordinate frame of the sensor and a coordinate frame of the preoperative model.

15. The system of claim 13, the system further comprising a field generator configured to generate an EM field, wherein the sensor is an EM sensor, and wherein the one or more processors are configured to execute the instructions to cause the system determine a position of the EM sensor within the EM field.

16. The system of claim 13, wherein the sensor is a shape sensing fiber.

17. The system of claim 13, wherein the one or more processors are configured to execute the instructions to cause the system to move the instrument within the luminal network.

18. The system of claim 13, wherein the one or more processors are configured to execute the instructions to cause the system to at least:

detect when the position of the distal end of the instrument is moved into the mapped portion of the luminal network; and

stop displaying the visual indicia when the distal end of the instrument is positioned inside the mapped portion of the luminal network.

19. A method of determining a navigation path of an instrument within a luminal network, the method comprising:

displaying, on a user interface, a preoperative model corresponding to a mapped portion of a luminal network;

determining a position of a distal end of an instrument within the luminal network relative to the mapped portion of the luminal network;

moving the instrument within the luminal network;

determining when the distal end of the instrument has been advanced past the mapped portion of the luminal network into an unmapped portion of the luminal network and entering a path tracing mode; and

when in the path tracing mode, displaying visual indicia of a path of the distal end of the instrument in the unmapped portion of the luminal network relative to the preoperative model of the mapped portion of the luminal network.

20. The method of claim 19, further comprising:

detecting when the position of the distal end of the instrument is moved into the mapped portion of the luminal network; and

stopping displaying the visual indicia when the distal end of the instrument is positioned inside the mapped portion of the luminal network.

21. The method of claim 19, further comprising adjusting a frequency of the visual indicia based on a distance traveled by the instrument between the visual indicia.

22. The method of claim 19, further comprising adjusting a frequency of the visual indicia based on a time elapsed between the visual indicia.

23. The method of claim 19, further comprising associating vision data with the visual indicia, wherein the vision data comprises an image received from an imaging device on the distal end of the instrument.

24. A non-transitory computer readable storage medium having stored thereon instructions that, when executed, cause a processor of a device to at least:

display, on a user display, a preoperative model corresponding to a mapped portion of a luminal network of a patient;

determine a position of a distal end of an instrument that is positioned within the luminal network relative to the preoperative model;

detect when the position of the distal end of the instrument is moved beyond the mapped portion of the luminal network; and

display, on the user display, visual indicia indicative of the position of the distal end of the instrument with respect to the displayed preoperative model when the distal end of the instrument is positioned outside the mapped portion of the luminal network.

25. The non-transitory computer readable storage medium of claim 24, wherein the instructions are configured to cause the processor of the device to:

detect when the position of the distal end of the instrument is moved into the mapped portion of the luminal network; and

stop displaying the visual indicia when the distal end of the instrument is positioned inside the mapped portion of the luminal network.

26. The non-transitory computer readable storage medium of claim 24, wherein the instructions are configured to cause the processor of the device to determine the position of the distal end of the instrument based on a plurality of navigation modalities when the instrument is positioned within the mapped portion of the preoperative model.

27. The non-transitory computer readable storage medium of claim 26, wherein the plurality of navigation modalities comprise a plurality of preoperative model data, vision data, position sensor data, and robotic command and kinematics data.

28. The non-transitory computer readable storage medium of claim 26, wherein the instructions are configured to cause the processor of the device to determine the position of the distal end of the instrument based on less than the plurality of navigation modalities when the instrument is positioned outside the mapped portion of the preoperative model.

29. The non-transitory computer readable storage medium of claim 24, wherein the instructions are configured to cause the processor of the device to determine the position of the distal end of the instrument based EM data received from an EM sensor when the instrument is positioned outside the mapped portion of the preoperative model.

30. The non-transitory computer readable storage medium of claim 24, wherein the instructions are configured to cause the processor of the device to determine the position of the distal end of the instrument based on a combination of vision data and robotic command and kinematics data when the instrument is positioned outside the mapped portion of the preoperative model.