Virtual endoscopy with improved image segmentation and lesion detection

• US 7,747,055 B1
• Filed: 11/24/1999
• Issued: 06/29/2010
• Est. Priority Date: 11/25/1998
• Status: Expired due to Fees

• Current Assignee: Wake Forest University Health Sciences
• Original Assignee: Wake Forest University Health Sciences
• Inventors: Stelts, David R., Ge, Yaorong, Ahn, David K., Salido, Tiffany W., Vining, David J., Hemler, Paul F., Hunt, Gordon W.
• Primary Examiner(s): Casler; Brian
• Assistant Examiner(s): Roy; Baisakhi
• Application Number:
  US09/856,683
• Time in Patent Office: 3,870 Days
• Field of Search: 600/407, 600/410, 600/416, 600/425, 600/424, 128/920, 345/419, 345/420, 345/421, 345/422, 345/423, 345/427, 382/131, 382/130
• US Class Current: 382/131
• CPC Class Codes: A61B 6/466 adapted to display 3D data
• CPC Class Codes: A61B 8/483 involving the acquisition of a 3D volume of data
• CPC Class Codes: G06T 15/08 Volume rendering
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10072 Tomographic images
• CPC Class Codes: G06T 2207/30028 Colon; Small intestine
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 50/50 for simulation or modelling of medical disorders
• CPC Class Codes: G16Z 99/00 Subject matter not provided for in other main groups of this subclass

Abstract

A system, and computer implemented method are provided for interactively displaying three-dimensional structures. Three-dimensional volume data (34) is formed from a series of two-dimensional images (33) representing at least one physical property associated with the three-dimensional structure, such as a body organ having a lumen. A wire frame model of a selected region of interest is generated (38b). The wireframe model is then deformed or reshaped to more accurately represent the region of interest (40b). Vertices of the wire frame model may be grouped into regions having a characteristic indicating abnormal structure, such as a lesion. Finally, the deformed wire frame model may be rendered in an interactive three-dimensional display.

Claims

1. A system for interactively displaying three dimensional structures comprising:

a. volume formation means for forming a three-dimensional volume of data from a series of two-dimensional images representing at least one physical property associated with a three dimensional body;

b. segmentation means for segmenting a region of interest from the volume of data based on selected values of the physical property representing the region of interest;

c. means for producing a wireframe model of the segmented region of interest, the wireframe model comprising a plurality of vertices, each vertex having a coordinate position;

d. means for refining the wireframe model by adjusting the coordinate positions of the vertices to more accurately represent the region of interest; and

e. rendering means for rendering the wireframe model in an interactive three-dimensional display, therein producing a virtual three-dimensional environment,

wherein the segmentation means approximates the region of interest and the refinement means adjusts the coordinate positions of the vertices along a normal vector associated with each vertex.

2. The system of claim 1 wherein the refinement means adjusts the coordinate positions of the vertices along a normal vector associated with each vertex as a function of a surface curvature at the coordinate position.

3. The system of claim 1 wherein the refinement means adjusts the coordinate positions of the vertices along a normal vector associated with each vertex by determining an inner wall position and an outer wall position from a first and a second degree derivative associated with the coordinate positions of the vertices.

4. The system of claim 1 wherein the segmentation means, the wireframe producing means, and the refining means are located remotely from the volume formation means.

5. A system for interactively displaying three dimensional structures comprising:

a. volume formation means for forming a three-dimensional volume of data from a series of two-dimensional images representing at least one physical property associated with a three dimensional body;

b. segmentation means for segmenting a region of interest from the volume of data based on selected values of the physical property representing the region of interest;

c. isosurface creation means for creating an isosurface of the segmented region of interest;

d. model means for producing a wireframe model of the segmented region of interest; and

e. rendering means for rendering the wireframe model in an interactive three-dimensional display, therein producing a virtual three dimensional environment,

wherein the isosurface creation means comprises means for identifying a seed voxel from the segmented region of interest and means for producing an isosurface patch associated with the seed voxel.

6. The system of claim 5 wherein the segmentation means, the isosurface creation means, and the wireframe producing means are located remotely from the volume formation means.

7. A system for interactively displaying a three-dimensional rendering of a structure having a lumen and indicating regions having abnormal structure comprising:

a. volume formation means for forming a three-dimensional volume of data from a series of two-dimensional images representing at least one physical property associated with the three-dimensional structure;

b. segmentation means for segmenting a region of interest from the volume of data based on a selected value of the physical property representing the region of interest;

c. wireframe generation means for generating a wireframe model of the segmented region of interest;

d. vertex grouping means for grouping the vertices of the wireframe model into regions having abnormal structure;

e. analyzing means for analyzing the regions having abnormal structure; and

f. rendering means for rendering the wireframe model in an interactive three-dimensional display to indicate the regions having abnormal structure.

8. The system of claim 7 wherein the analyzing means comprises means for evaluating two principal curvatures associated with each point in an elliptical region.

9. The system of claim 8 wherein the analyzing means comprises a means for identifying ridges.

10. The system of claim 8 wherein the segmentation means, the wireframe generating means, the vertex grouping means, and the analyzing means are located remotely from the volume formation means.

11. A method for guiding biopsy of a tissue by accessing the tissue via a three-dimensional structure having a lumen comprising the steps of:

a. forming a three-dimensional volume of data from a series of two-dimensional images representing at least one physical property associated with the three-dimensional structure and at least one physical property associated with the tissue;

b. segmenting a first region of interest from the volume of data based on a selected value of the physical property associated with the three-dimensional structure;

c. creating a first surface of the first region of interest;

d. generating a first wireframe model of the surface of the first region of interest;

e. segmenting a second region of interest from the volume of data based on a selected value of the physical property associated with the tissue;

f. creating a second surface of the second region of interest;

g. generating a second wireframe model of the isosurface of the second region of interest;

h. rendering the first and second wireframe models in an interactive three-dimensional environment;

i. inserting a simulated biopsy needle within the lumen of the three-dimensional structure;

j. determining the position of the simulated biopsy needle within the three-dimensional structure;

k. displaying the rendered, interactive three-dimensional display from a perspective indicating the position of the biopsy needle within the three-dimensional structure; and

l. generating a zone around the simulated tissue and issuing an indicator when the tip of the simulated biopsy needle is within the zone.

12. The method of claim 11 comprising the step of indicating the position of the simulated biopsy needle relative to the position of the tissue.

13. The method of claim 12 wherein the indicating step comprises the step of determining a distance and a trajectory from the simulated biopsy needle to the simulated tissue.

14. A system for interactively displaying three dimensional structures comprising:

a. volume formation means for forming a three-dimensional volume of data from a series of two-dimensional images representing at least one physical property associated with a three-dimensional body;

b. modeling means for producing a wireframe model of a selected region of interest, the wireframe model comprising a plurality of vertices, each vertex having a coordinate position;

c. refinement means for refining the wireframe model by adjusting the coordinate positions of the vertices to more accurately represent the region of interest; and

d. rendering means for rendering the wireframe model in an interactive three-dimensional display, therein producing a virtual three-dimensional environment,

wherein the refinement means adjusts the coordinate positions of vertices of the wireframe model along a normal vector associated with each vertex at a rate that is a function of a surface curvature at the coordinate position.

15. The system of claim 14 wherein the refinement means adjusts the coordinate positions of the vertices along a normal vector associated with each vertex by determining an inner wall position and an outer wall position from an analysis of intensity profiles of voxel values along the normal vector.

16. The system of claim 14 wherein the modeling means, and the refining means are located remotely from the volume formation means.

17. A system for interactively displaying three dimensional structures comprising:

a. volume formation means for forming a three-dimensional volume of data from a series of two-dimensional images representing at least one physical property associated with a three dimensional body;

b. wireframe producing means for producing a wireframe model of a selected region of interest of the three-dimensional volume;

c. deforming means for deforming the wireframe model using a geometric deformable model to more accurately represent the region of interest; and

d. rendering means for rendering the wireframe model in an interactive three-dimensional display, therein producing a virtual three-dimensional environment.

18. A system for interactively displaying a three-dimensional rendering of a structure having a lumen and indicating regions having abnormal structure comprising:

a. volume formation means for forming a three-dimensional volume of data from a series of two-dimensional images representing at least one physical property associated with the three-dimensional structure;

b. wireframe generation means for generating a wireframe model of a selected region of interest;

c. vertex grouping means for grouping the vertices of the wireframe model into regions having abnormal structure;

d. analyzing means for analyzing the regions having abnormal structure; and

e. rendering means for rendering the wireframe model in an interactive three-dimensional display to indicate the regions having abnormal structure.

19. The system of claim 18 wherein the analyzing means comprises means for evaluating two principal curvatures associated with each point in an elliptical region.

20. The system of claim 19 wherein the analyzing means comprises a means for identifying ridges.

21. The system of claim 20 comprising wireframe deforming means for deforming the wireframe model to more accurately represent the region of interest.

22. A method for guiding biopsy of a tissue by accessing the tissue via a three-dimensional structure having a lumen comprising the steps of:

a. forming a three-dimensional volume of data from a series of two-dimensional images representing at least one physical property associated with the three-dimensional structure and at least one physical property associated with the tissue;

b. generating a wireframe model of a selected region of interest;

c. rendering the wireframe model in an interactive three-dimensional display;

d. inserting a simulated biopsy needle within the lumen of the three-dimensional structure;

e. determining the position of the simulated biopsy needle within the three-dimensional structure;

f. displaying the rendered, interactive three-dimensional display from a perspective indicating the position of the simulated biopsy needle within the three-dimensional structure;

g. generating a zone around the tissue and issuing an indicator when a tip of the biopsy needle is within the zone.

23. The method of claim 22 comprising the step of indicating the position of the simulated biopsy needle relative to the position of the tissue.

24. The method of claim 23 wherein the indicating step comprises the step of determining a distance and a trajectory from the simulated biopsy needle to the tissue.

25. The method of claim 22 comprising an image acquisition step wherein a series of two-dimensional images representing at least one physical property associated with the structure is acquired, wherein the image acquisition step comprises a patient preparation step.

26. The method of claim 25 wherein the patient preparation step comprises the step of administering to a patient a nonionic intravenous iodinated contrast bolus.

27. The method of claim 25 wherein the patient preparation step comprises the step of administering to a patient an oral contrast agent for opacifying stool and water.

28. A method for guiding biopsy of a tissue by accessing the tissue via a three-dimensional structure having a lumen comprising the steps of:

a. providing a three-dimensional volume of data representing at least one physical property associated with the three-dimensional structure;

b. isolating a first region of interest of the structure having the lumen;

c. rendering the first region of interest in an interactive three-dimensional display;

d. inserting a simulated biopsy needle within the lumen of the rendered first region of interest;

e. determining the position of the simulated biopsy needle within the rendered first region of interest having the lumen;

f. displaying the simulated biopsy needle within the rendered first region of interest having the lumen;

g. determining the position of the needle relative to a selected target zone;

h. generating a ready-to-strike indicator if the needle is within a desired location relative to the selected target zone; and

i. calculating a movement pathway for the needle to strike the target zone if the needle is not within a desired location relative to the selected target zone.

29. The method of claim 28 comprising the step of isolating the first region of interest from the volume of data based on a selected value of the physical property associated with the three-dimensional structure.

30. The method of claim 29 wherein the step of isolating the first region of interest comprises segmenting the first region of interest.

31. The method of any one of claims 28-30 comprising:

providing a three-dimensional volume of data representing at least one physical property associated with the tissue;

isolating a second region of interest of the structure having the tissue;

rendering the second region of interest in the interactive three-dimensional display; and

creating at least one target zone of needle placement at a selected site of the rendered tissue.

32. The method of claim 31 comprising the step of isolating the second region of interest from the volume of data based on a selected value of the physical property associated with the tissue.

33. The method of claim 32 wherein the step of isolating the second region of interest comprises segmenting the second region of interest.

34. The method of claim 28 comprising:

providing a three-dimensional volume of data representing at least one physical property associated with mediastinal tissue;

isolating a second region of interest of the structure having the mediastinal tissue;

rendering the second region of interest in the interactive three-dimensional display; and

positioning the needle so as to avoid the rendered mediastinal tissue.

35. A system for interactively displaying three dimensional structures comprising:

a. volume formation means for forming a three-dimensional volume of data representing at least one physical property associated with a three dimensional structure;

b. a wireframe generator for generating a wireframe model having vertices of a region of interest representing the three-dimensional structure;

c. refinement means for refining the wireframe model by using a geometric deformable model;

d. a renderer for rendering the wireframe model in an interactive three-dimensional display;

e. a vertex grouper for grouping the vertices of the wireframe model into regions having abnormal structure; and

f. an analyzer for analyzing the regions having abnormal structure.

36. The system of claim 35 wherein the renderer produces a virtual three-dimensional environment.

37. The system of claim 35 comprising an isolator for isolating the region of interest from the volume of data based on selected values of the physical property representing the region of interest.

38. The system of claim 37 wherein the renderer produces a virtual three-dimensional environment.

39. The system of claim 37 wherein the isolator comprises a surface model identification means for identifying a surface model and wherein the wireframe generator is adapted to create a wireframe model of the surface model.

40. The system of claim 37 wherein the isolator comprises a segmentation producer for segmenting the region of interest from the volume of data.

41. The system of claim 40 wherein the isolator comprises an isosurface creation means for creating an isosurface of the isolated region of interest.

42. The system of claim 41 wherein the wireframe generator is adapted to create a wireframe model of the isosurface.

43. The system of claim 35 wherein the vertex grouper comprises:

a. a mean curvature calculator for calculating a mean curvature at each of a selected number of vertices;

b. a gaussian curvature calculator for calculating a gaussian curvature at each of the selected number of vertices; and

c. classification means for classifying the selected vertices according to the mean curvatures and the gaussian curvatures of the vertices.

44. A system for interactively displaying three dimensional structures comprising:

a. volume formation means for forming a three-dimensional volume of data representing at least one physical property associated with a three dimensional structure;

b. a wireframe generator for generating a wireframe model having a plurality of vertices to represent a region of interest of the three-dimensional structure;

c. a vertex grouper for collecting adjoining vertices having similar classifications based on mean and gaussian curvatures into patches of similar vertex classification; and

d. a renderer for rendering the wireframe model in an interactive three-dimensional display.

45. The system of claim 44 wherein the renderer produces a virtual three-dimensional environment.

46. The system of claim 44 comprising an isolator for isolating the region of interest from the volume of data based on selected values of the physical property representing the region of interest.

47. The system of claim 46 wherein the renderer produces a virtual three-dimensional environment.

48. The system of claim 46 or 47 wherein the isolator comprises a segmentation producer for segmenting the region of interest from the volume of data.

49. The system of claim 46 wherein the isolator comprises a surface model identification means for identifying a surface model and wherein the wireframe generator is adapted to create a wireframe model of the surface model.

50. The system of claim 46 wherein the isolator comprises an isosurface creation means for creating an isosurface of the isolated region of interest.

51. The system of claim 44 wherein the vertex grouper comprises:

a mean curvature calculator for calculating a mean curvature at each of a selected number of vertices;

a gaussian curvature calculator for calculating a gaussian curvature at each of the selected number of vertices; and

classification means for classifying the selected vertices according to the mean curvatures and the gaussian curvatures of the vertices.