Method for cell envelope segmentation and visualisation

• US 10,235,759 B2
• Filed: 04/01/2015
• Issued: 03/19/2019
• Est. Priority Date: 04/01/2014
• Status: Active Grant

• Current Assignee: Stryker European Operations Limited (Stryker Corporation)
• Original Assignee: Scopis GmbH (Stryker Corporation)
• Inventors: Kosmecki, Bartosz, Oezbek, Christopher, Winne, Christian
• Primary Examiner(s): Couso, Jose L
• Application Number:
  US15/300,414
• Publication Number:
  US 20170148173A1
• Time in Patent Office: 1,448 Days
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/10081 Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/149 involving deformable models, e.g. active contour models

Abstract

The present invention relates to a method for the segmentation and visualization of cell envelopes. A method is provided for the segmentation and visualization of cavities or orifices of the human body, comprising the steps of manual pre-segmentation of cells by defining enclosing geometric primitives in a 3D patient image for generating initial cell envelopes; analysing the cell anatomy within pre-segmented geometric primitives; using the result of the analysis for adjustment of the cell envelope and visualising the cell envelopes.

Claims

1. A method for a user to generate a reliable computer-aided segmentation and visualization of a plurality of cells of the paranasal sinuses for guiding a surgical instrument wherein the method is at least partially executed on an information processing system, said method comprising the steps of:

a. receiving 3D patient image data of the paranasal sinuses;

b. receiving a manual pre-segmentation of the plurality of cells, the manual pre-segmentation of cells comprising a user input defining a geometric primitive in a slice image of the 3D patient image data for generating a cell envelope, the geometric primitive comprising a shape selected from a group comprising a cuboid, sphere, cylinder and ellipse and each having a position, size, and depth defined by the user;

c. adjusting the cell envelope by optimizing the shape, position, orientation, and/or size of the geometric primitive based on an analysis of a cell anatomy within the pre-segmented geometric primitive without additional input from the user;

d. determining, by the information processing system, a natural drainage pathway between, along, and/or around the plurality of segmented cells using pathfinding algorithms based on the 3D patient image data;

e. displaying the cell envelope and the natural drainage pathway relative to the 3D patient image data; and

f. displaying the natural drainage pathway as linked cylindrical segments in augmented reality views.

2. The method of claim 1, wherein the step of determining the natural drainage pathway between the plurality of segmented cells further comprises conducting a global cell analysis, wherein the global cell analysis is configured to identify anatomical aberrations or pathologies.

3. The method of claim 1, wherein the step of determining the natural drainage pathway between the plurality of segmented cells further comprises conducting a global cell analysis, wherein the global cell analysis is configured to identify critical anatomical sites such as nerves and/or blood vessels.

4. The method of claim 1, wherein the step of determining the natural drainage pathway between the plurality of segmented cells further comprises conducting a global cell analysis, wherein the global cell analysis is configured to determine a cell linkage based on shortest path, connectivity, or anatomical aspects.

5. The method of claim 4, wherein the determined cell linkage are visualized as entry and exit locations into neighboring cells or at least one surgical pathway is visualized as a trajectory between an entry and exit location of the plurality of segmented cells.

6. The method of claim 1, wherein properties of the plurality of cells are determined and displayed to the user including the cell volume, filling, disease status, drainage pathway and orifices.

7. The method of claim 1, further comprising communicating the determined natural drainage pathway to the surgical navigation system configured to navigate the surgical instrument based on the determined natural drainage pathway.

8. The method of claim 1, further comprising the step of labelling of the segmented cells based on receiving a manual input or automatic detection by said system.

9. The method of claim 1, wherein the step of determining the natural drainage pathway between the plurality of segmented cells further comprises conducting a global cell analysis, wherein the global cell analysis is configured to identify additional inter-space cells.

10. A method for a user to generate a reliable computer-aided segmentation and visualization of a plurality of cells of the paranasal sinuses for guiding a surgical instrument wherein the method is at least partially executed on an information processing system, said method comprising the steps of:

a. receiving 3D patient image data of the paranasal sinuses;

b. receiving a manual pre-segmentation of the plurality of cells, the manual pre-segmentation of cells comprising a user input defining a geometric primitive in a slice image of the 3D patient image data for generating a cell envelope, the geometric primitive comprising a shape selected from a group comprising a cuboid, sphere, cylinder and ellipse and each having a position, size, and depth defined by the user;

c. adjusting the cell envelope by optimizing the shape, position, orientation, and/or size of the geometric primitive based on an analysis of a cell anatomy within the pre-segmented geometric primitive without additional input from the user;

d. determining, by the information processing system, a natural drainage pathway between, along, and/or around segmented cells using pathfinding algorithms based on the 3D patient image data;

e. displaying the cell envelope and the natural drainage pathway relative to the 3D patient image data; and

wherein the adjusted cell envelope or the determined natural drainage pathway are displayed pre- or intra-operatively in a virtual endoscopic view or intra-operatively in an augmented-reality-view.

11. A method for a user to generate a reliable computer-aided segmentation and visualization of a plurality of cells of the paranasal sinuses for guiding a surgical instrument wherein the method is at least partially executed on an information processing system, said method comprising the steps of:

a. receiving 3D patient image data of the paranasal sinuses;

b. receiving a manual pre-segmentation of the plurality of cells, the manual pre-segmentation of cells comprising a user input defining a geometric primitive in a slice image of the 3D patient image data for generating a cell envelope, the geometric primitive comprising a shape selected from a group comprising a cuboid, sphere, cylinder and ellipse and each having a position, size, and depth defined by the user;

c. adjusting the cell envelope by optimizing the shape, position, orientation, and/or size of the geometric primitive based on an analysis of a cell anatomy within the pre-segmented geometric primitive without additional input from the user;

d. determining, by the information processing system, a natural drainage pathway between, along, and/or around segmented cells using pathfinding algorithms based on the 3D patient image data;

e. displaying the determined natural drainage pathway relative to the 3D patient image data; and

wherein the step of displaying further comprises presenting the determined natural drainage pathway in a plurality of slice images, in a 3D view, in a virtual endoscopic view, or in an endoscopic augmented reality view as either;

linked line segments,

splines,

inked curved cylindrical segments,

meshes, or

voxels.

12. The method of claim 11, wherein the step of displaying comprises presenting the determined natural drainage pathway in the plurality of slice images, wherein each of the slice images comprises a sectional illustration of the pre-segmented geometric primitive and/or the adjusted cell envelope in the plane of an individual slice image of the plurality of slice images, wherein the sectional illustration comprises the boundaries of the geometric primitive and/or the cell envelope or its sectional area marked opaque or semi-transparent.

13. The method of claim 11, wherein the step of displaying comprises presenting the determined natural drainage pathway in the plurality of slice images, wherein each of the slice images comprises highlighting of a currently selected cell by using a different line-type, color, line width, or opacity to show the currently selected cell relative to other cells.

14. The method of claim 11, wherein the step of displaying comprises presenting the determined natural drainage pathway in the plurality of slice images, wherein each of the slice images comprises displaying the determined natural drainage pathway as an intersection of the determined natural drainage pathway with the plane of each of the slice images in the form of a point or highlighted area.

15. The method of claim 11, wherein the step of displaying further comprises presenting the determined natural drainage pathway in the 3D view, wherein the 3D view comprises:

displaying a perspective or orthographic representation of the geometric primitive and/or the adjusted cell envelope in a 3D view of the nasal cavities;

displaying of labels to identify the cells, wherein the labels comprise textual information, numbers, or symbols;

or

hiding a surface of the 3D image data to allow the relative position of the plurality of segmented cells to one another to be visualized by the user.

16. The method of claim 11, wherein the step of displaying comprises presenting the determined natural drainage pathway in a plurality of slice images, in a 3D view, in a virtual endoscopic view, or in an endoscopic augmented reality view as linked line segments.

17. The method of claim 11, wherein the step of displaying comprises presenting the determined natural drainage pathway in a plurality of slice images, in a 3D view, in a virtual endoscopic view, or in an endoscopic augmented reality view as splines.

18. The method of claim 11, wherein the step of displaying comprises presenting the determined natural drainage pathway in a plurality of slice images, in a 3D view, in a virtual endoscopic view, or in an endoscopic augmented reality view as linked curved cylindrical segments.

19. The method of claim 11, wherein the step of displaying comprises presenting the determined natural drainage pathway in a plurality of slice images, in a 3D view, in a virtual endoscopic view, or in an endoscopic augmented reality view as meshes.

20. The method of claim 11, wherein the step of displaying comprises presenting the determined natural drainage pathway in a plurality of slice images, in a 3D view, in a virtual endoscopic view, or in an endoscopic augmented reality view as voxels.