Navigation of tubular networks

• US 10,169,875 B2
• Filed: 08/04/2017
• Issued: 01/01/2019
• Est. Priority Date: 09/18/2015
• Status: Active Grant

• Current Assignee: Auris Health, Inc. (Johnson & Johnson)
• Original Assignee: Auris Health, Inc. (Johnson & Johnson)
• Inventors: Mintz, David S., Ghoreyshi, Atiyeh, Jeevan, Prasanth, Xu, Yiliang, Yang, Gehua, Leotta, Matthew Joseph, Stewart, Charles V.
• Primary Examiner(s): Pontius, James M
• Application Number:
  US15/669,258
• Publication Number:
  US 20170365055A1
• Time in Patent Office: 515 Days
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 1/00147 Holding or positioning arrangements
• CPC Class Codes: A61B 1/00149 using articulated arms
• CPC Class Codes: A61B 1/0016 using motor drive units
• CPC Class Codes: A61B 1/00194 adapted for three-dimensional imaging
• CPC Class Codes: A61B 1/2676 Bronchoscopes
• CPC Class Codes: A61B 2017/00809 Lung operations
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2034/2048 using an accelerometer or inertia sensor
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2034/252 indicating steps of a surgical procedure
• CPC Class Codes: A61B 2034/301 for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes
• CPC Class Codes: A61B 2090/3614 using optical fibre
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 5/061 Determining position of a probe within the body employing means separate from the probe, e.g. sensing internal probe position employing impedance electrodes on the surface of the body A61B1/00158 takes precedence
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 90/30 Devices for illuminating a surgical field, the devices having an interrelation with other surgical devices or with a surgical procedure
• CPC Class Codes: G06T 15/205 : Image-based rendering
• CPC Class Codes: G06T 17/00 : Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2207/10068 : Endoscopic image
• CPC Class Codes: G06T 2207/10081 : Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/30061 : Lung
• CPC Class Codes: G06T 2207/30196 : Human being; Person
• CPC Class Codes: G06T 7/0012 : Biomedical image inspection
• CPC Class Codes: G06T 7/149 : involving deformable models, e.g. active contour models
• CPC Class Codes: G06T 7/248 : involving reference images or patches
• CPC Class Codes: G06T 7/32 : using correlation-based methods
• CPC Class Codes: G06T 7/73 : using feature-based methods

Abstract

Methods and apparatuses provide improved navigation through tubular networks such as lung airways by providing improved estimation of location and orientation information of a medical instrument (e.g., an endoscope) within the tubular network. Various input data such as image data, EM data, and robot data are used by different algorithms to estimate the state of the medical instrument, and the state information is used to locate a specific site within a tubular network and/or to determine navigation information for what positions/orientations the medical instrument should travel through to arrive at the specific site. Probability distributions together with confidence values are generated corresponding to different algorithms are used to determine the medical instrument'"'"'s estimated state.

Claims

1. A system comprising:

a set of one or more processors; and

a computer readable-medium that includes computer program instructions that, when executed, cause the set of processors to;

access a first set of sensor data regarding physical manipulation of an elongated medical instrument inserted into a tubular network of a patient, the first set of sensor data being derived from at least a first set of one or more sensors located at a proximal portion of the of the elongated medical instrument;

access a second set of sensor data regarding the positioning of the medical elongated medical instrument, the second set of sensor data being derived from at least a second set of one or more sensors located within the elongated medical instrument, the second set of sensors being different from the first set of sensors;

determine a first estimated state regarding the physical manipulation of the elongated medical instrument based on the first set of sensor data;

determine a second estimate state regarding the physical manipulation of the elongated medical instrument based on the second set of sensor data; and

determine an estimated state of the elongated medical instrument based on the first estimated state and the second estimated state.

2. The system of claim 1 wherein instructions, when executed, cause the set of one or more processors to access a third set of sensor data regarding the positioning of the medical elongated medical instrument, the third set of sensor data being derived from at least a third set of one or more sensors, the third set of sensors being different from the first and second set of sensors.

3. The system of claim 2, wherein the instructions, when executed, cause the set of one or more processors to determine a third estimate state regarding the elongated medical instrument based on the third set of sensor data.

4. The system of claim 1, wherein the instructions, when executed, cause the set of one or more processors to associate the first estimated state with a first confidence value, to associate the second estimated state with a second confidence value, and to determine the estimated state of the elongated medical instrument based on the first estimated state, the second estimated state, the first confidence value and the second confidence value.

5. The system of claim 4, wherein the confidence value for a corresponding one of the estimated states represents a degree of confidence in the determination of the corresponding estimated state.

6. The system of claim 1, wherein accessing the first set of sensor data regarding physical manipulation of the elongated medical instrument inserted into a tubular network of a patient comprises accessing robotic data regarding the physical manipulation of an elongated medical instrument inserted into a tubular network of a patient.

7. The system of claim 6, wherein the robotic data includes at least one of:

pitch, roll, and yaw of a tip of the instrument, the pitch, the roll, and the yaw based on retraction of at least one of a plurality of cables of the elongated medical instrument and insertion distance of the elongated medical instrument into the tubular network.

8. The system of claim 1, wherein accessing the first set of sensor data regarding physical manipulation of the elongated medical instrument inserted into a tubular network of a patient comprises accessing image data captured by an imaging device located proximal to an instrument tip of the elongated medical instrument.

9. The system of claim 8, wherein the image data is based on a series of images captured sequentially in time.

10. The system of claim 1, wherein accessing the second set of sensor data regarding the positioning of the medical elongated medical instrument comprises accessing electromagnetic (EM) data captured using an EM sensor located proximal to a tip of the medical elongated medical instrument and using at least on external EM sensor or EM generated located external to the patient.

11. The system of claim 1, wherein the elongated medical instrument is a flexible endoscope comprising an endolumenal structure and a plurality of cables that are configured to change a direction of a tip of the instrument.

12. The system of claim 1, wherein the estimated state indicates location of the instrument tip within the tubular network.

13. The system of claim 1, wherein the instructions, when executed, cause the set of one or more processors to access a prior estimated state for medical elongated medical instrument based on data captured at a prior instant in time and determe at least one of the first and second estimated states based on the prior estimated state.

14. The system of claim 1, wherein determining the second estimated state further comprises performing registration of an EM system to a 3D model, the EM system including coordinates based on received EM data and the 3D model including coordinates based on a 3D model of the tubular network;

and the second estimated state based on the performed registration.

15. The system of claim 1, wherein the 3D model of the tubular network is generated based on computerized axial tomography (CT) scans of the tubular network.

16. The system of claim 1, wherein determining second estimated state further comprises:

determining an estimate of position and orientation of the instrument tip relative to possible branches within the tubular network.

17. The system of claim 1, wherein the second estimated state is represented as a discrete probability distribution, each of values of the probability distribution indicating a likelihood of a tip of an instrument being inside a corresponding branch among the possible branches.

18. The system of claim 1, wherein determining the first estimated state further comprises:

measuring movement of the elongated medical instrument within the tubular network based on accessed image data; and

generating the first estimated state based on the measured movement.

19. A non-transitory computer readable storage medium comprising computer program instructions that when executed by a set of one or more processors cause the set of processors to:

access first set of data from a first set of one or more sensors regarding physical manipulation of an elongated medical instrument inserted into a tubular network of a patient;

access second set data captured by as second set of one or more sensors located proximal to an instrument tip of the elongated medical instrument;

determine a first estimated state for the instrument tip based on the first set of data;

determine a second estimated state for the instrument tip based on the second set of data; and

determine an estimated state for the instrument tip based on the first and second estimated states.

20. The non-transitory computer readable storage medium of claim 19 wherein the instructions, when executed, cause the at least one processor to access a third set of data captured using a third set of sensors comprising one more sensors located proximal to an instrument tip of the elongated medical instrument.

21. The non-transitory computer readable storage medium of claim 20 wherein the instructions, when executed, cause the at least one processor to determine a third estimated state for the instrument tip based on the third set of data.

22. The non-transitory computer readable storage medium of claim 21 wherein the instructions, when executed, cause the at least one processor to determine the estimated state for the instrument tip based on the first, second and third estimated states.

23. The non-transitory computer readable storage medium of claim 19 wherein the instructions, when executed, cause the at least one processor when accessing first set of data to access robotic data regarding the physical manipulation of an elongated medical instrument inserted into a tubular network of a patient.

24. The non-transitory computer readable storage medium of claim 23 wherein the robotic data includes at least one of:

pitch, roll, and yaw of a tip of the instrument, the pitch, the roll, and the yaw based on retraction of at least one of a plurality of cables of the elongated medical instrument and insertion distance of the elongated medical instrument into the tubular network.

25. The non-transitory computer readable storage medium of claim 19 wherein the instructions, when executed, cause the at least one processor when accessing the first set of sensor data to access image data captured by an imaging device located proximal to an instrument tip of the elongated medical instrument.

26. The non-transitory computer readable storage medium of claim 25 wherein the image data is based on a series of images captured sequentially in time.

27. The non-transitory computer readable storage medium of claim 19 wherein the instructions, when executed, cause the at least one processor when accessing the second set of sensor data to access electromagnetic (EM) data captured using an EM sensor located proximal to a tip of the medical elongated medical instrument and using at least on external EM sensor or EM generated located external to the patient.

28. The non-transitory computer readable storage medium of claim 19 wherein the estimated state indicates location of the instrument tip within a tubular network.

29. The non-transitory computer readable storage medium of claim 19 wherein the instructions, when executed, cause the at least one processor to associate the first estimated state with a first confidence value and to associate the second estimated state with a second confidence value.

30. The non-transitory computer readable storage medium of claim 29 wherein the instructions, when executed, cause the at least one processor to determine an estimated state for the instrument tip based on the first and second estimated states and the first and second confidence values.