AUTOMATIC DEPTH SCROLLING AND ORIENTATION ADJUSTMENT FOR SEMI-AUTOMATED PATH PLANNING

• US 20140344742A1
• Filed: 11/28/2012
• Published: 11/20/2014
• Est. Priority Date: 12/03/2011
• Status: Active Grant

• Current Assignee: Koninklijke Philips N.V.
• Original Assignee: Koninklijke Philips N.V.
• Inventors: Wiemker, Rafael, Xu, Sheng, Klinder, Tobias, Bergtholdt, Martin, Lu, Kongkuo
• Granted Patent: US 10,758,212 B2
• Time in Patent Office: Days
• US Class Current: 715/771
• CPC Class Codes: A61B 10/0233 Pointed or sharp biopsy instruments
• CPC Class Codes: A61B 2034/101 Computer-aided simulation of surgical operations
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G16H 40/40 for the management of medical equipment or devices, e.g. scheduling maintenance or upgrades

Abstract

A planning tool, system and method include a processor (114) and memory (116) coupled to the processor which stores a planning module (144). A user interface (120) is coupled to the processor and configured to permit a user to select a path through a pathway system (148). The planning module is configured to upload one or more slices of an image volume (111) corresponding to a user-controlled cursor point (108) guided using the user interface such that as the path is navigated the one or more slices are updated in accordance with a depth of the cursor point in the path.

Claims

1. A planning system, comprising:

a processor (114);

memory (116) coupled to the processor and storing a planning module (144); and

a user interface (120) coupled to the processor and configured to permit a user to select a path through a pathway system (148);

the planning module configured to upload one or more slices of an image volume (111) corresponding to a user-controlled cursor point (108) guided using the user interface such that as the path is navigated the one or more slices are updated in accordance with a depth of the cursor point in the path.

2. The system as recited in claim 1, wherein the planning module (144) is configured to generate a trace line to indicate the path.

3. The system as recited in claim 1, wherein the pathway system (148) includes a lung and the path is determined between an entry point and a target within the lung.

4. The system as recited in claim 1, wherein the user-controlled cursor point (108) is centered within a depth dimension of an endoluminal structure and the planning module (144) checks the depth with images slices to determine whether the image slice should be updated.

5. The system as recited in claim 1, further comprising an image processing module (115) configured to perform a local extraction of an endoluminal structure of the pathway in two directions about a cursor location.

6. The system as recited in claim 1, wherein the memory (116) stores all three-dimensional positions along the path created by a drag of the cursor point to employ as input for path planning.

7. The system as recited in claim 1, wherein the image volume (111) includes pre-operatively collected images.

8. A system having operatively coupled viewports, comprising:

a processor (114);

memory (116) coupled to the processor and storing an image processing module (115);

a graphical user interface (106) coupled to the processor and configured to permit a user to select a path through a pathway system (148);

the image processing module (115) being configured to render;

an endoluminal view (404) of the pathway;

one or more other views (402, 406, 408) of an image volume (111); and

a virtual line (410) to provide a spatial reference in all selected views including the endoluminal view and the one or more other views;

a user-controlled cursor point (412) configured to guide the endoluminal view wherein the cursor point is employed to permit updates in all the selected views corresponding to a selection update in the cursor point such that image information surrounding the cursor point in other views is concurrently viewable in the user interface.

9. The system as recited in claim 8, wherein the one or more views (402, 406, 408) includes one or more of a standard view and a reformatted view.

10. The system as recited in claim 8, wherein a reformatted view includes one of a view parallel to the virtual line or a view perpendicular to the virtual line.

11. The system as recited in claim 8, wherein the virtual line (410) is advanced or retracted in the endoluminal view by advancing or withdrawing a pointing control device.

12. The system as recited in claim 8, wherein the endoluminal view (404) is employed to determine a needle insertion point in a wall of the endoluminal structure and at least one other view is employed to determine a structure through the wall not visible in the endoluminal view.

13. The system as recited in claim 8, wherein the virtual line (410) is indicated in all the selected views.

14. The system as recited in claim 8, further comprising a pointing control device (108) configured to change a camera angle for at least one of the views.

15. The system as recited in claim 8, wherein the graphical user interface (106) includes a display (118) such that a plurality of panes corresponding to multiple views (124) is concurrently displayed.

16. A method for planning a procedure, comprising:

locating (506) an end point in an image volume of a pathway structure, the image volume comprised of stacked slices along a depth;

selecting (508) a start point in the image volume of the pathway structure;

extending (510) a path along the pathway structure;

updating (512) image slices of the pathway structure along the path in accordance with a depth of the pathway structure corresponding with the image slices;

navigating (516) along the pathway to the end point; and

storing (518) the path for creating a plan for a procedure.

17. The method as recited in claim 16, further comprising segmenting (504) the pathway structure in pieces corresponding with a current location of a cursor.

18. The method as recited in claim 16, wherein the pathway structure includes an airway of a lung and the path is determined between a target as the start point and a trachea as the end point.

19. The method as recited in claim 16, further comprising computing (514) a center location for an endoluminal structure in the path and comparing the center location to a trace of the path to determine whether the stacked slice being viewed should be updated.

20. The method as recited in claim 16, further comprising performing (505) a local extraction of an endoluminal structure of the pathway structure in two directions about a cursor location.

21. The method as recited in claim 16, wherein the image volume includes pre-operatively collected images (111).

22. A method for operatively coupling viewports, comprising:

generating (604) an endoluminal view of a pathway structure in a graphical user interface configured to permit a user to select a path through a pathway system;

generating (608) one or more other views of an image volume; and

generating (612) a virtual line to provide a spatial reference in a plurality of views including the endoluminal view and the one or more other views such that the virtual line is configured to guide the endoluminal view and is employed to provide a reference to permit updates in the other views corresponding to a user selected position update of the virtual line such that image information surrounding the virtual line in the other views is concurrently viewable in the user interface.

23. The method as recited in claim 22, wherein the one or more views includes one or more of a standard view and a reformatted view.

24. The method as recited in claim 22, wherein a reformatted view includes one of a view parallel to the virtual line or a view perpendicular to the virtual line.

25. The method as recited in claim 22, wherein the virtual line is advanced or retracted in the endoluminal view by advancing or withdrawing a pointing control device.

26. The method as recited in claim 22, wherein the endoluminal view is employed (618) to determine a needle insertion point in a wall of the endoluminal structure and at least one other view is employed to determine a structure through the wall not visible in the endoluminal view.

27. The method as recited in claim 22, wherein the virtual line is indicated (622) in all selected views.

28. The method as recited in claim 22, further comprising changing (624) a camera angle for at least one of the views using a pointing control device.

29. The method as recited in claim 22, wherein the endoluminal and other views are concurrently displayed in a plurality of panes.