MEDICAL IMAGE REPORTING SYSTEM AND METHOD

• US 20100310146A1
• Filed: 02/16/2009
• Published: 12/09/2010
• Est. Priority Date: 02/14/2008
• Status: Active Grant

• Current Assignee: Penn State Research Foundation (Pennsylvania State University)
• Original Assignee: Penn State Research Foundation (Pennsylvania State University)
• Inventors: Gibbs, Jason D., Higgins, Williams E., Lu, Kongkuo, Graham, Michael W., Yu, Kun-Chang
• Granted Patent: US 9,672,631 B2
• Time in Patent Office: Days
• US Class Current: 382/131
• CPC Class Codes: G06T 2207/20044 Skeletonization; Medial axis transform
• CPC Class Codes: G06T 2207/30061 Lung
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/162 involving graph-based methods

Abstract

This invention relates generally to medical imaging and, in particular, to a method and system for reconstructing a model path through a branched tubular organ. Novel methodologies and systems segment and define accurate endoluminal surfaces in airway trees, including small peripheral bronchi. An automatic algorithm is described that searches the entire lung volume for airway branches and poses airway-tree segmentation as a global graph-theoretic optimization problem. A suite of interactive segmentation tools for cleaning and extending critical areas of the automatically segmented result is disclosed. A model path is reconstructed through the airway tree.

Claims

1. A method of reconstructing a model of a path through a branching tubular organ, drawing upon a three-dimensional (3D) digital image of the surrounding anatomy and comprising the steps of:

computing a segmentation of the organ using the digital image;

identifying a region of interest (ROI) associated with the anatomy;

selecting the path originating at the beginning of the branching organ, passing through branches of the organ, and leading to the ROI;

adding branches to the organ segmentation;

constructing accurate endoluminal surfaces of the organ'"'"'s branches; and

producing a model of the path containing accurate endoluminal surfaces at all branch junctions encountered along the path.

2. The method of claim 1, wherein the step of computing a segmentation of the organ comprises:

computing an initial coarse segmentation of the organ using the digital image;

identifying a set of candidate branch sections;

combining groups of candidate branch sections into a set of disjoint candidate branch segments;

linking nearby branch segments using connection functions; and

calculating an optimal reconstruction of the organ using the coarse segmentation, the candidate branch segments, and the connection functions.

3. The method of claim 1, wherein the step of adding branches comprises adding branches to the organ model along the path.

4. The method of claim 2, wherein the step of identifying a set of candidate peripheral branch segments includes the steps of:

scanning the digital image for short branch sections; and

connecting sections that share a common boundary.

5. The method of claim 4, wherein the step of connecting branch sections that share a common boundary includes the use of smooth interpolated surfaces.

6. The method of claim 2, wherein the step of identifying branch sections includes the use of a global graph partitioning algorithm.

7. The method of claim 1, wherein the 3D digital image is a computed-tomography image of the chest.

8. The method of claim 1, wherein the branching tubular organ is an airway tree.

9. The method of claim 1, wherein the branching tubular organ is a vascular system.

10. The method of claim 1, wherein the model of the path enables guidance of endoscopy to a diagnostic ROI.

11. The method of claim 1, wherein the endoluminal surfaces are derived from a suitable topological dilation of the organ segmentation.

12. The method of claim 1, wherein the endoluminal surfaces are computed from locally defined isosurfaces.

13. The method of claim 1, wherein the endoluminal surfaces are computed from a combination of the methods of claims 11 and 12.

14. The method of claim 11, wherein the topological dilation is determined by a linear programming algorithm and the linear programming algorithm satisfies topological and smoothness constraints.

30. The method of claim 1 wherein global information in the image is used to construct the model.

31. The method of claim 1 wherein said connection functions are smooth interpolated parametric connection functions.

32. The method of claim 1 further comprising removing sections from the organ model.

34. The method of claim 1 further comprising displaying said at least one lymph node station in a view.

35. The method of claim 1 wherein said anatomical structure comprise at least one of the patient'"'"'s airway tree, aorta, pulmonary artery, lungs, vertebrae, and sternum from the image data.

36. The method of claim 3 wherein said anatomical structure comprise the airway tree, and said method further comprising calculating centerlines through the airways of the airway tree.

37. The method of claim 4 further comprising labeling individual airways of the airway tree.

38. The method of claim 1 wherein said calculating the at least one lymph node station is performed entirely automatically.

39. The method of claim 1 wherein the calculating step comprises calculating a plurality of Mountain Stations.

40. The method of claim 1 further comprising accepting input regarding a lymph node of the station.

41. The method of claim 8 further comprising calculating the lymph node within the station based on said input.

42. The method of claim 9 further comprising displaying said lymph node.

43. The method of claim 5 wherein said lymph node station is station M7, and said anatomical cues comprise a main carina, an inferior limit of a bronchus intermedius, a right limit of the right main bronchus, and left limit of the left main bronchus.

44. The method of claim 2 comprising displaying said station in a 3D view and a 2D multiplanar formatted image section.

45. The method of claim 1 comprising adjusting the anatomical cues.

46. The method of claim 2 comprising adjusting the view of the station.

47. The method of claim 7 further comprising calculating at least 10 Mountain Stations, and indicating which of said Mountain Stations are accessible via a bronchoscope.

48. The method of claim 1 wherein said image data is 3D image data.

65. The method of claim 31 wherein said extracting step comprises extracting one or more landmarks from the group consisting of the main carina location, trachea, major bronchi, and aortic arch.

66. The method of claim 31 comprising reviewing the lymph-node stations and existing lymph nodes in a processed MDCT chest scan with a 2D section tool and a 3D surface rendering tool.

67. The method of claim 31 further comprising adding comments on at least one of said lymph-node stations and extracted lymph nodes.

68. The method of claim 34 comprising providing a printed case report.

70. The method of claim 1, wherein the three-dimensional digital image of the anatomy is derived from a multidetector computed tomography scan.

71. The method of claim 1, wherein the three-dimensional digital image is derived from a magnetic resonance imaging scan.

72. The method of claim 1, including the step of drawing the ROI using a three-dimensional livewire method.

73. The method of claim 1, wherein the step of automatically generating a path to the site includes the step of defining the interior surfaces of the organ.

74. The method of claim 1, wherein the step of automatically generating a path to the site includes the step of extracting the medial axes (centerlines) of branches within the organ.

75. The method of claim 1, wherein the final view of the report includes unique iconic or numerical information at the final location along the path.

76. The method of claim 1, wherein the step of outputting a report describing the path includes generating a static report presenting relevant visual and quantitative data along the path.

77. The method of claim 1, wherein the step of outputting a report describing the path includes the generation of a dynamic report providing an interactive virtual-reality movie preview of the procedure.

78. The method of claim 1, wherein the report depicts organ branch and/or bifurcation views encountered along the path.

79. The method of claim 1, wherein the organ is a tracheobronchial tree.

80. The method of claim 1, wherein the organ is a coronary arterial tree.

81. The method of claim 1, wherein the organ is a gastrointestinal tract or colon.

83. The system of claim 14, wherein the three-dimensional digital image of the organ is derived from a multidetector computed tomography scan.

84. The system of claim 14, wherein the three-dimensional digital image is derived from a magnetic resonance imaging scan.

85. The system of claim 14, wherein the processor is operative to automatically defining the interior surfaces of the organ to generate a path to the ROI.

86. The system of claim 14, wherein the processor is operative to automatically extract the medial axes (centerlines) of branches within the organ to generate a path to the site.

87. The system of claim 14, including a display showing unique iconic or numerical information at the final location along the path.

89. The system of claim 14, including a display showing a static report presenting relevant visual and quantitative data along the path.

90. The system of claim 14, including a display showing an interactive virtual-reality movie preview of the procedure.

91. The system of claim 14, wherein the report depicts organ branch and/or bifurcation views encountered along the path.

92. The system of claim 14, wherein the organ is a tracheobronchial tree.

93. The system of claim 14, wherein the organ is a coronary arterial tree.

94. The system of claim 14, wherein the organ is a gastrointestinal tract or colon.

95. The method of claim 1, wherein the step of outputting a report describing the path includes generating a report for a portable display device, wherein said device is one of a tablet computer, notebook computer, PDA, and smartphone.

15. A system for reconstructing a model of a path through a branching tubular organ, comprising the steps of:

an input for receiving a 3D digital image of the surrounding anatomy;

a processor operative to perform the following functions;

a) automatically computing a segmentation of the organ using the digital image;

b) receiving input for identifying a region of interest (ROI) associated with the anatomy;

c) selecting the path originating at the beginning of the branching organ, passing through branches of the organ, and leading to the ROI;

d) receiving input for adding branches to the organ segmentation along the path;

e) constructing accurate endoluminal surfaces of the organ'"'"'s branches; and

f) producing a model of the path containing accurate endoluminal surfaces at all branch junctions encountered along the path.

16. The system of claim 15, wherein the automatic segmentation method includes the steps of:

computing an initial coarse segmentation of the organ using the digital image;

identifying a set of candidate branch sections;

combining groups of candidate branch sections into a set of disjoint candidate branch segments;

linking nearby branch segments using connection functions; and

calculating an optimal reconstruction of the organ using the coarse segmentation, the candidate branch segments, and the connection functions.

17. The system of claim 15, including an input for adding new candidate branches to the organ.

18. The system of claim 15, wherein the processor is operative to identify a set of candidate branch segments, including:

scanning the digital image for short branch sections; and

connecting sections that share a common boundary.

19. The system of claim 18, wherein the processor is operative to connect branch sections that share a common boundary using smooth interpolated surfaces.

20. The system of claim 18, wherein the processor is operative to determine valid branches among the candidates using a global graph partitioning algorithm.

21. The system of claim 15, wherein the 3D digital image is a computed-tomography image of the chest.

22. The system of claim 15, wherein the branching tubular organ is an airway tree.

23. The system of claim 15, wherein the branching tubular organ is a vascular system.

24. The system of claim 15, wherein the model of the path enables guidance of endoscopy to a diagnostic ROI.

25. The system of claim 15, wherein the processor is operative to determine endoluminal surfaces from a suitable topological dilation of the organ segmentation.

26. The system of claim 15, wherein the endoluminal surfaces are computed from locally defined isosurfaces.

27. The system of claim 15, wherein the endoluminal surfaces are computed from a combination of the methods of claims 25 and 26.

28. The system of claim 25, wherein the topological dilation is determined by a linear programming algorithm and the linear programming algorithm satisfies topological and smoothness constraints.

29. The system of claim 15 further comprising a display in communication with the processor for displaying the 3D image and one or more of the results obtained through functions a)-f).

50. The method of claim 17 wherein said area is a Mountain Station.

51. The method of claim 17 wherein said automatically calculating is carried out entirely automatically.

52. The method of claim 17 wherein the anatomical cue is calculated based on image data selected from the group consisting of 2D and 3D image data.

54. The system of claim 21 further comprising a display in communication with said processor and operative to display a view of the lymph node station in 3D.

55. The system of claim 21 wherein said processor is operative to reconstruct at least one of the anatomical structures selected from the group consisting of an airway tree, aorta, pulmonary artery, lungs, vertebrae, and sternum.

56. The system of claim 21 wherein said processor is operative to calculate a 3D geometrical shape that defines Mountain station M7.

57. The system of claim 21 further comprising a node tool to accept input regarding a lymph node within said lymph node station, and said processor being operative to calculate said lymph node in 3D.

58. The system of claim 21 further comprising a cue tool to adjust said at least one anatomical cue.

59. The system of claim 22 comprising a display tool to adjust the view of the station.

60. The system of claim 21 wherein said anatomical structure comprises the airway tree, and the processor is operative to calculate centerlines of airways of said airway tree.

61. The system of claim 25 wherein said processor being operative to calculate a route to said lymph node from a first location.

63. The system of claim 29 wherein said processor is configured to display the route in at least one view.

88. The system of claim 19, wherein the display device is one of a tablet computer, notebook computer, PDA, and smartphone.

33. A method for automatically identifying at least one lymph node station in a thoracic region of a patient, said method comprising the steps of:

calculating a 3D model of at least one anatomical structure in the chest based on image data;

calculating at least one anatomical cue derived from said at least one anatomical structure; and

calculating at least one lymph node station based on said image data and using said at least one anatomical cue wherein said calculating at least one lymph node station is performed automatically.

49. A method for automatically identifying at least one lymph node zone in the thoracic region comprising the steps of:

identifying at least one known anatomical cue; and

automatically calculating at least one lymph node zone using said at least one anatomical cue.

53. A computing system for calculating at least one lymph node station based on image data of a patient'"'"'s thoracic region, said system comprising:

a memory storing said image data;

a processor in communication with said memory, the processor being operative to;

a) calculate a 3D model of at least one anatomical structure in the chest of the patient;

b) calculate at least one anatomical cue derived from said at least one anatomical structure; and

c) calculate said at least one lymph node station based on at least one anatomical cue.

64. A lymph node station mapping method based on image data of a patient of the thoracic region, said method comprising:

processing the image data, said processing step comprising (a) automatically defining at least one anatomical structure from the group consisting of;

airway tree, airway-tree centerlines, aorta, pulmonary artery, lungs, major skeletal structures, and major-airway labels; and

(b) defining criteria for MDCT-based station definition based on geometric and anatomical criteria defined by the Mountain system;

defining a landmark and a station comprising (a) extracting a landmark for a lymph-node station definition;

(b) defining Mountain stations based on said criteria and the extracting step; and

(c) demarcating each station with at least one 3D volumetric region; and

visualizing the station and lymph-node definition, said visualization step comprising (a) manually refining the region occupied by each lymph-node station using at least one tool; and

(b) segmenting visible lymph nodes in the defined station regions.

69. An automatic method of producing a report of a path through a hollow body organ, comprising the steps of:

receiving previously acquired three-dimensional (3D) image data of the anatomy, including predefined endoluminal surfaces of the organ;

identifying at least one site of interest associated with the anatomy;

automatically generating a path to the site; and

outputting a report describing a selected characteristic, or characteristics, along the path.

82. A system for producing a report of a path through a hollow body organ, comprising:

a user input for identifying at least one site of interest associated with the anatomy on a three-dimensional digital image of the anatomy, including predefined endoluminal surfaces; and

a processor for automatically generating a path to the site and a report describing the path.

96. Any method disclosed herein.

97. Any article disclosed herein.

98. Any system disclosed herein.