Dynamic 3D lung map view for tool navigation inside the lung

• US 10,660,708 B2
• Filed: 10/01/2018
• Issued: 05/26/2020
• Est. Priority Date: 07/02/2014
• Status: Active Grant

• Current Assignee: Covidien LP (Medtronic Plc)
• Original Assignee: Covidien LP (Medtronic Plc)
• Inventors: Weingarten, Oren P., Barak, Ron
• Primary Examiner(s): Ansari, Tahmina N
• Application Number:
  US16/148,174
• Publication Number:
  US 20190038359A1
• Time in Patent Office: 603 Days
• Field of Search: None
• CPC Class Codes: A61B 2017/00809 Lung operations
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 90/37 Surgical systems with images on a monitor during operation
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/30064 Lung nodule
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling
• CPC Class Codes: G06T 7/0012 Biomedical image inspection

Abstract

A method for implementing a dynamic three-dimensional lung map view for navigating a probe inside a patient'"'"'s lungs includes loading a navigation plan into a navigation system, the navigation plan including a planned pathway shown in a 3D model generated from a plurality of CT images, inserting the probe into a patient'"'"'s airways, registering a sensed location of the probe with the planned pathway, selecting a target in the navigation plan, presenting a view of the 3D model showing the planned pathway and indicating the sensed location of the probe, navigating the probe through the airways of the patient'"'"'s lungs toward the target, iteratively adjusting the presented view of the 3D model showing the planned pathway based on the sensed location of the probe, and updating the presented view by removing at least a part of an object forming part of the 3D model.

Claims

1. A method comprising:

displaying a three-dimensional (3D) model of a lung generated from a plurality of CT images, the displayed 3D model including a lung object or a target forming part of the 3D model; and

updating the display of the 3D model by ghosting at least part of another object, which forms part of the displayed 3D model and obstructs a view of at least part of the lung object or the target.

2. The method according to claim 1, wherein ghosting the at least part of the another object forming part of the displayed 3D model includes displaying the at least part of the another object forming part of the displayed 3D model with a higher level of transparency than the lung object or the target of the 3D model.

3. The method according to claim 1, wherein ghosting the at least part of the another object forming part of the displayed 3D model includes removing from the display the at least part of the another object forming part of the displayed 3D model.

4. The method according to claim 1, further comprising:

displaying, in the 3D model, a planned pathway and a sensed location of a probe, the probe including a location sensor; and

adjusting the display of the 3D model and the planned pathway when the sensed location of the probe changes.

5. The method according to claim 4, further comprising:

registering the sensed location of the probe with the planned pathway; and

selecting another target in a navigation plan.

6. The method according to claim 4, wherein adjusting the display of the 3D model and the planned pathway includes zooming in when at least one of:

the probe approaches another target;

or

a diameter of an airway of the lung within which the probe is sensed to be located is less than a predetermined threshold.

7. The method according to claim 4, wherein adjusting the display of the 3D model and the planned pathway includes changing at least one of a viewpoint and a zoom to maximize a separation of bifurcated airways.

8. The method according to claim 4, wherein adjusting the display of the 3D model and the planned pathway includes displaying a viewpoint from a distal portion of the probe in a direction of a most recent movement of the probe.

9. The method according to claim 4, wherein adjusting the display of the 3D model and the planned pathway includes displaying a viewpoint from a distal portion of the probe in a direction opposite a most recent movement of the probe.

10. The method according to claim 4, wherein adjusting the display of the 3D model and the planned pathway includes displaying a viewpoint from a distal portion of the probe in a direction orthogonal to a most recent movement of the probe.

11. The method according to claim 4, wherein adjusting the display of the 3D model and the planned pathway includes displaying a viewpoint from a distal portion of the probe in a direction along the planned pathway to another target.

12. The method according to claim 4, further comprising providing an alert when at least one of:

the probe is approaching a pleura;

the probe is approaching major blood vessels;

or

the sensed location of the probe is off of the planned pathway.

13. The method according to claim 1, further comprising ghosting:

at least part of a second lung object or another target that is outside of a region of interest;

at least part of the another object or the target that is obstructing a view of a probe;

at least part of the lung object or the target that is not relevant to a sensed location of the probe;

or

at least part of the lung object or the target that is not relevant to a current selected state of a navigation system of the probe.

14. The method according to claim 1, wherein the 3D model of the lung is generated from the plurality of CT images by smoothing gaps in a representation of airways formed from the plurality of CT images.

15. The method according to claim 4, wherein adjusting the display of the 3D model and the planned pathway includes rotating a viewpoint around a focal point to improve 3D perception of the sensed location of the probe in relation to the 3D model.

16. A method comprising:

generating a three-dimensional (3D) model of a patient based on a plurality of CT images of the patient, the generated 3D model including an anatomical object or a target forming part of the 3D model;

displaying the generated 3D model; and

updating the display of the 3D model by ghosting at least part of another object, which forms part of the displayed 3D model to and obstructs a view of the at least part of the anatomical object or the target.

17. The method according to claim 16, further comprising:

displaying, in the 3D model, a planned pathway and a sensed location of a probe;

adjusting the display of the 3D model and the planned pathway when the sensed location of the probe changes;

registering the sensed location of the probe with the planned pathway; and

selectively displaying targets in the 3D model based on the sensed location of the probe.

18. A method comprising:

displaying a three-dimensional (3D) model of a lung and at least one target of a navigation plan generated from a plurality of CT images, the displayed 3D model including a lung object forming part of the 3D model; and

updating the displayed 3D model by;

ghosting at least part of the lung object, which forms part of the displayed 3D model and obstructs a view of the at least part of the lung object; and

selectively displaying the at least one target in the displayed 3D model.

19. The method according to claim 18, further comprising displaying a planned pathway to the at least one target.

20. The method according to claim 18, further comprising displaying another target upon navigating to the at least one target.