Navigation of tubular networks

• US 10,482,599 B2
• Filed: 04/01/2019
• Issued: 11/19/2019
• Est. Priority Date: 09/18/2015
• Status: Active Grant

• Current Assignee: Auris Health, Inc. (Johnson & Johnson)
• Original Assignee: Auris Health, Inc. (Johnson & Johnson)
• Inventors: Mintz, David S., Ghoreyshi, Atiyeh, Jeevan, Prasanth, Xu, Yiliang, Yang, Gehua, Leotta, Matthew Joseph, Stewart, Charles V.
• Primary Examiner(s): Pontius, James M
• Application Number:
  US16/372,093
• Publication Number:
  US 20190228528A1
• Time in Patent Office: 232 Days
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 1/00147 Holding or positioning arrangements
• CPC Class Codes: A61B 1/00149 using articulated arms
• CPC Class Codes: A61B 1/0016 using motor drive units
• CPC Class Codes: A61B 1/00194 adapted for three-dimensional imaging
• CPC Class Codes: A61B 1/2676 Bronchoscopes
• CPC Class Codes: A61B 2017/00809 Lung operations
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2034/2048 using an accelerometer or inertia sensor
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2034/252 indicating steps of a surgical procedure
• CPC Class Codes: A61B 2034/301 for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes
• CPC Class Codes: A61B 2090/3614 using optical fibre
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 5/061 Determining position of a probe within the body employing means separate from the probe, e.g. sensing internal probe position employing impedance electrodes on the surface of the body A61B1/00158 takes precedence
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 90/30 Devices for illuminating a surgical field, the devices having an interrelation with other surgical devices or with a surgical procedure
• CPC Class Codes: G06T 15/205 : Image-based rendering
• CPC Class Codes: G06T 17/00 : Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2207/10068 : Endoscopic image
• CPC Class Codes: G06T 2207/10081 : Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/30061 : Lung
• CPC Class Codes: G06T 2207/30196 : Human being; Person
• CPC Class Codes: G06T 7/0012 : Biomedical image inspection
• CPC Class Codes: G06T 7/149 : involving deformable models, e.g. active contour models
• CPC Class Codes: G06T 7/248 : involving reference images or patches
• CPC Class Codes: G06T 7/32 : using correlation-based methods
• CPC Class Codes: G06T 7/73 : using feature-based methods

Abstract

Methods and apparatuses provide improved navigation through tubular networks such as lung airways by providing improved estimation of location and orientation information of a medical instrument (e.g., an endoscope) within the tubular network. Various input data such as image data, EM data, and robot data are used by different algorithms to estimate the state of the medical instrument, and the state information is used to locate a specific site within a tubular network and/or to determine navigation information for what positions/orientations the medical instrument should travel through to arrive at the specific site. Probability distributions together with confidence values are generated corresponding to different algorithms are used to determine the medical instrument'"'"'s estimated state.

Claims

1. A robotic medical system for bronchoscopy, the system comprising:

a robotic arm comprising a plurality of segments, the robotic arm extending from a base;

a robotically-controllable bronchoscope removably coupled to the robotic arm, the bronchoscope configured for insertion into a bronchial network of a patient, wherein a distal end of the bronchoscope is articulable; and

a command console comprising a command module configured to allow an operator to remotely control the bronchoscope, wherein the command module is configured to allow the operator to remotely articulate the distal end bronchoscope, and wherein the command module comprises a trackball;

a display;

at least one computer-readable medium having stored thereon instructions and at least one processor in communication with the at least one computer-readable medium and configured to execute the instructions to cause a navigation module to;

display a model of the bronchial network;

access a selected target location in the bronchial network;

access a determined path from an entry point to the target location;

receive input data related to a position of the bronchoscope from a plurality of data sources;

determine the position of the bronchoscope within the bronchial network based on the input data from one or more of the plurality of data sources;

display the determined position of the bronchoscope relative to the displayed model of the bronchial network; and

display navigational guidance on the display, the navigational guidance comprising the determined path to the target location, a distance to the target location, a location of the bronchoscope relative to the model of the bronchial network, and depth data.

2. The system of claim 1, wherein the navigation module is configured to:

display the target location on the displayed model of the bronchial network; and

display the target location relative to a position and orientation of the bronchoscope.

3. The system of claim 1, wherein the plurality of data sources comprises at least one of:

position sensor data received from a position sensor on the bronchoscope,

image data received from an imaging device on the bronchoscope,

robot data comprising command data for instructing the system to change the orientation of the bronchoscope, and

model data.

4. The system of claim 3, wherein the robot data comprises at least one of:

a pitch, a roll, and a yaw of the bronchoscope; and

an insertion distance of the bronchoscope into the bronchial network.

5. The system of claim 3, wherein the model data comprises a computer model of the bronchial network generated based on computerized axial tomography (CT) scans of the tubular network.

6. The system of claim 1, wherein the determined path comprises a path along centerlines of lumens of the model.

7. The system of claim 1, wherein the base comprises a cart with wheels.

8. The system of claim 1, wherein the robotic arm comprises at least one set-up joint including a brake and counter-balance to maintain a set-up position of the robotic arm.

9. The system of claim 1, wherein the navigation module is configured to display a computer model of the bronchoscope relative to the model of the bronchial network.

10. The system of claim 1, wherein the display comprises two or more display modules.

11. The system of claim 1, wherein the bronchoscope includes an imaging device, and wherein a biopsy tool or surgical instrument can be inserted into a working channel of the bronchoscope without removing the imaging device.

12. A method for robotically-assisted bronchoscopy, the method comprising:

displaying a model of a bronchial network;

accessing a selected target location in the bronchial network;

accessing a determined path from an entry point of the bronchial network to the target location;

inserting a robotically-controllable bronchoscope into a bronchial network of a patient with a robotic arm comprising a plurality of segments, the robotic arm extending from a base;

receiving input data relating to a position of the bronchoscope from a plurality of data sources;

determining the position of the bronchoscope within the bronchial network based on the input data from one or more of the plurality of data sources;

displaying the determined position of the bronchoscope relative to the displayed model of the bronchial network;

displaying navigational guidance comprising;

the determined path to the target location, a distance to the target location, a location of the bronchoscope relative to the model of the bronchial network, and depth data.

13. The method of claim 12, further comprising:

displaying the target location on the displayed model of the bronchial network.

14. The method of claim 13, further comprising displaying the target location relative to a position and orientation of the bronchoscope.

15. The method of claim 12, wherein the plurality of data sources comprises at least one of:

position sensor data received from a position sensor on the bronchoscope,

image data received from an imaging device on the bronchoscope,

robot data comprising command data for instructing the system to change the orientation of the bronchoscope, and

model data.

16. The method of claim 15, wherein the robot data comprises at least one of:

a pitch, a roll, and a yaw of the bronchoscope; and

an insertion distance of the bronchoscope into the bronchial network.

17. The method of claim 15, further comprising:

generating the computer model based on computerized axial tomography (CT) scans of the tubular network.

18. The method of claim 12, wherein accessing the determined path from the entry point of the bronchial network to the target location comprises determining a path along centerlines of lumens of the model.

19. The method of claim 12, further comprising displaying a computer model of the bronchoscope relative to the displayed model of the bronchial network.

20. The method of claim 12, further comprising receiving a user command from a command console including a control module configured to allow an operator to remotely control the bronchoscope.

21. The method of claim 20, wherein the control module comprises a trackball.

22. The method of claim 12, further comprising inserting a biopsy tool or surgical instrument into a working channel of the bronchoscope, wherein the bronchoscope includes an imaging device, and wherein the biopsy tool or surgical instrument is inserted into the working channel without removing the imaging device.