Automatic depth scrolling and orientation adjustment for semi-automated path planning

• US 10,758,212 B2
• Filed: 11/28/2012
• Issued: 09/01/2020
• Est. Priority Date: 12/03/2011
• Status: Active Grant

• Current Assignee: Koninklijke Philips N.V.
• Original Assignee: Koninklijke Philips N.V.
• Inventors: Wiemker, Rafael, Lu, Kongkuo, Xu, Sheng, Klinder, Tobias, Bergtholdt, Martin
• Primary Examiner(s): Savla, Arpan P.
• Assistant Examiner(s): Patel, Parmanand D
• Application Number:
  US14/361,884
• Publication Number:
  US 20140344742A1
• Time in Patent Office: 2,834 Days
• CPC Class Codes: A61B 10/0233 Pointed or sharp biopsy instruments
• CPC Class Codes: A61B 2034/101 Computer-aided simulation of surgical operations
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G16H 40/40 for the management of medical equipment or devices, e.g. scheduling maintenance or upgrades

Abstract

A planning tool, system and method include a processor (114) and memory (116) coupled to the processor which stores a planning module (144). A user interface (120) is coupled to the processor and configured to permit a user to select a path through a pathway system (148). The planning module is configured to upload one or more slices of an image volume (111) corresponding to a user-controlled cursor point (108) guided using the user interface such that as the path is navigated the one or more slices are updated in accordance with a depth of the cursor point in the path.

Claims

1. A planning system, comprising:

a processor;

memory coupled to the processor and storing a planning module; and

a user interface coupled to the processor and configured to permit a user to select a path through a pathway system by controlling a displayed cursor point;

the planning module configured to upload two or more slices of an image volume corresponding to the displayed cursor point and provide an automatic adaptation of depth, position and orientation, such that as the path is navigated by controlling the displayed cursor point, the two or more slices are updated continuously to automatically change a view plane and orientation of the slices based on changes in a position of the displayed cursor point in response to a user interaction changing the position of the displayed cursor by moving a mouse or pointing device of the graphical user interface.

2. The system as recited in claim 1, wherein the planning module is configured to generate a trace line to indicate the path.

3. The system as recited in claim 1, wherein the pathway system includes a lung and the path is determined between an entry point and a target within the lung.

4. The system as recited in claim 1, wherein the displayed cursor point is centered within a depth dimension of an endoluminal structure and the planning module checks the depth with images slices to determine whether the image slice should be updated.

5. The system as recited in claim 1, further comprising an image processing module configured to perform a local extraction of an endoluminal structure of the pathway in two directions about a cursor location.

6. The system as recited in claim 1, wherein the memory stores all three-dimensional positions along the path created by a drag of the cursor point to employ as input for path planning.

7. The system as recited in claim 1, wherein the image volume includes pre-operatively collected images.

8. A system having operatively coupled viewports, comprising:

a processor;

memory coupled to the processor and storing an image processing module;

a graphical user interface coupled to the processor and configured to permit a user to select a path through a pathway system;

the image processing module being configured to render;

an endoluminal view of the pathway, and

one or more other views of an image volume; and

a virtual line to provide a spatial reference in all of the views including the endoluminal view and the one or more other views;

a user-controlled cursor point configured to guide the endoluminal view wherein when the cursor point is moved in any of the views in response to a user interaction with the system via the graphical user interface all of the other views are automatically updated to change a view plane and orientation in the other views such that image information surrounding the user-controlled cursor point in the other views is concurrently viewable in the user interface.

9. The system as recited in claim 8, wherein the one or more views includes one or more of a standard view and a reformatted view.

10. The system as recited in claim 8, wherein a reformatted view includes one of a view parallel to the virtual line or a view perpendicular to the virtual line.

11. The system as recited in claim 8, wherein the virtual line is advanced or retracted in the endoluminal view by advancing or withdrawing a pointing control device.

12. The system as recited in claim 8, wherein the endoluminal view is employed to determine a needle insertion point in a wall of the endoluminal structure and at least one other view is employed to determine a structure through the wall not visible in the endoluminal view.

13. The system as recited in claim 8, wherein the virtual line is indicated in all the selected views.

14. The system as recited in claim 8, further comprising a pointing control device configured to change a camera angle for at least one of the views.

15. The system as recited in claim 8, wherein the graphical user interface includes a display such that a plurality of panes corresponding to multiple views is concurrently displayed.

16. A method for planning a procedure, comprising:

locating an end point in an image volume of a pathway structure, the image volume comprised of stacked slices along a depth;

selecting a start point in the image volume of the pathway structure by controlling a displayed cursor point;

extending a path along the pathway structure;

updating image slices of the pathway structure along the path continuously to automatically change a view plane and orientation based on a position of the displayed cursor point in accordance with a depth of the pathway structure corresponding with the slices;

navigating along the pathway to the end point and providing an automatic adaptation of depth, position and orientation; and

storing the path for creating a plan for a procedure.

17. The method as recited in claim 16, further comprising segmenting the pathway structure in pieces corresponding with a current location of a cursor.

18. The method as recited in claim 16, wherein the pathway structure includes an airway of a lung and the path is determined between a target as the start point and a trachea as the end point.

19. The method as recited in claim 16, further comprising computing a center location for an endoluminal structure in the path and comparing the center location to a trace of the path to determine whether the stacked slice being viewed should be updated.

20. The method as recited in claim 16, further comprising performing a local extraction of an endoluminal structure of the pathway structure in two directions about a cursor location.

21. The method as recited in claim 16, wherein the image volume includes pre-operatively collected images.

22. A method for operatively coupling viewports, comprising:

generating an endoluminal view of a pathway structure in a graphical user interface configured to permit a user to select a path through a pathway system by controlling a displayed cursor point and providing an automatic adaptation of depth, position and orientation;

generating one or more other views of an image volume continuously;

using only a change of position of the displayed cursor point in any one of the views, in response to a user interaction changing the position of the displayed cursor in the any one view by moving a mouse or pointing device of the graphical user interface, to automatically change a view plane and orientation of all of the other views; and

;

generating a virtual line to provide a spatial reference in a plurality of views including the endoluminal view and the one or more other views such that the virtual line is configured to guide the endoluminal view and is employed to provide a reference to permit updates in the other views corresponding to a user selected position update of the virtual line such that image information surrounding the virtual line in the other views is concurrently viewable in the user interface.

23. The method as recited in claim 22, wherein the one or more views includes one or more of a standard view and a reformatted view.

24. The method as recited in claim 22, wherein a reformatted view includes one of a view parallel to the virtual line or a view perpendicular to the virtual line.

25. The method as recited in claim 22, wherein the virtual line is advanced or retracted in the endoluminal view by advancing or withdrawing a pointing control device.

26. The method as recited in claim 22, wherein the endoluminal view is employed to determine a needle insertion point in a wall of the endoluminal structure and at least one other view is employed to determine a structure through the wall not visible in the endoluminal view.

27. The method as recited in claim 22, wherein the virtual line is indicated in all selected views.

28. The method as recited in claim 22, further comprising changing a camera angle for at least one of the views using a pointing control device.

29. The method as recited in claim 22, wherein the endoluminal and other views are concurrently displayed in a plurality of panes.