Diagnostic endoscopic imaging support apparatus and method, and non-transitory computer readable medium on which is recorded diagnostic endoscopic imaging support program

• US 10,085,672 B2
• Filed: 07/22/2014
• Issued: 10/02/2018
• Est. Priority Date: 01/24/2012
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Masumoto, Jun
• Primary Examiner(s): Fernandez, Katherine
• Assistant Examiner(s): Yang, Yi-Shan
• Application Number:
  US14/337,867
• Publication Number:
  US 20140336501A1
• Time in Patent Office: 1,533 Days
• CPC Class Codes: A61B 1/00006 of control signals
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/00055 for alerting the user
• CPC Class Codes: A61B 1/041 Capsule endoscopes for imaging
• CPC Class Codes: A61B 1/2676 Bronchoscopes
• CPC Class Codes: A61B 1/31 for the rectum, e.g. proctoscopes, sigmoidoscopes , colonoscopes
• CPC Class Codes: A61B 5/055 involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
• CPC Class Codes: A61B 5/066 Superposing sensor position on an image of the patient, e.g. obtained by ultrasound or x-ray imaging
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/12 Arrangements for detecting or locating foreign bodies
• CPC Class Codes: G01R 33/285 Invasive instruments, e.g. catheters or biopsy needles, specially adapted for tracking, guiding or visualization by NMR

Abstract

A diagnostic endoscopic imaging support apparatus includes a three-dimensional image data obtaining section that obtains three-dimensional image data of a subject, a tubular tissue shape data obtaining section that extracts and obtains tubular tissue shape data representing a shape of a tubular tissue in the subject from the three-dimensional image data obtained by the three-dimensional image data obtaining section, an endoscope route data obtaining section that obtains endoscope route data representing a route of an endoscope inserted into the subject, and a matching section that performs matching between the tubular tissue shape data and the endoscope route data.

Claims

1. A diagnostic endoscopic imaging support apparatus, comprising:

a processor configured to execute a process, the process comprising;

obtaining three-dimensional image data of a subject;

extracting and obtaining tubular tissue shape data representing a shape of a tubular tissue in the subject from the three-dimensional image data;

obtaining endoscope route data representing a set of positions along a route of an endoscope inserted into the subject and including a position of a tip of the endoscope,

wherein the position of the tip the endoscope corresponds to a front-most position along said series of positions;

extracting, from the tubular tissue shape data and the endoscope route data, target points at which curvature of the shape of the tubular tissue indicated by the tubular tissue shape data and curvature of the route of the endoscope indicated by the endoscope route data commonly exceed a predetermined threshold;

using the extracted target points, performing matching between the shape of the tubular tissue represented by the tubular tissue shape data and the series of positions along the route of the endoscope including the position of the tip of the endoscope represented by the endoscope route data,

wherein, when the route of the endoscope in the subject is changed, a detection of a variation in the route of the endoscope is made and the matching is performed in real-time for the part of the route of the endoscope for which the variation has been detected, and reuses the result of previous matching for the remaining part of the route without performing the matching anew; and

transforming the series of positions along the route of the endoscope based on a result of the matching; and

a display configured to display the transformed series of positions along the route of the endoscope in the subject in an image obtained from the three-dimensional image data.

2. The diagnostic endoscopic imaging support apparatus as claimed in claim 1, wherein the image is a three-dimensional stereoscopic image.

3. The diagnostic endoscopic imaging support apparatus as claimed in claim 1, wherein the image is a cross-sectional image.

4. The diagnostic endoscopic imaging support apparatus as claimed in claim 1, wherein the process further comprises:

obtaining tree structure data as the tubular tissue shape data; and

performing matching using the tree structure data.

5. The diagnostic endoscopic imaging support apparatus as claimed in claim 4, wherein the process further comprises:

obtaining line segment structure data as the endoscope route data; and

performing matching of local maximum points of curvature between the line segment structure data and the tree structure data.

6. The diagnostic endoscopic imaging support apparatus as claimed in claim 1, wherein the process further comprises obtaining information of distance from the insertion opening of the endoscope to the tip of the endoscope inserted into the subject and performing the matching using the information of distance.

7. The diagnostic endoscopic imaging support apparatus as claimed in claim 2, wherein the display control section displays the tubular tissue as a surface model.

8. The diagnostic endoscopic imaging support apparatus as claimed in claim 2, wherein the display control section displays the tubular tissue as a voxel model.

9. The diagnostic endoscopic imaging support apparatus as claimed in claim 1, wherein the process further comprises:

obtaining tip position information of the endoscope in a coordinate of the three-dimensional image data based on a result of the matching;

obtaining virtual endoscopic image data virtually generated on the assumption that imaging is performed at the tip position of the endoscope based on the tip position information and the three-dimensional image data; and

displaying, on the display, a virtual endoscopic image based on the virtual endoscopic image data and a real endoscopic image actually captured by the endoscope.

10. The diagnostic endoscopic imaging support apparatus as claimed in claim 2, wherein the process further comprises:

obtaining tip position information of the endoscope in a coordinate of the three-dimensional image data based on a result of the matching;

obtaining virtual endoscopic image data virtually generated on the assumption that imaging is performed at the tip position of the endoscope based on the tip position information and the three-dimensional image data; and

displaying, on the display, a virtual endoscopic image based on the virtual endoscopic image data and a real endoscopic image actually captured by the endoscope.

11. The diagnostic endoscopic imaging support apparatus as claimed in claim 9, wherein the process further comprises displaying, on the display, the virtual endoscopic image and the real endoscopic image side-by-side.

12. The diagnostic endoscopic imaging support apparatus as claimed in claim 9, wherein the process further comprises displaying, on the display, the virtual endoscopic image within the display screen of the real endoscopic image.

13. The diagnostic endoscopic imaging support apparatus as claimed in claim 9, wherein the process further comprises performing, on the display, a blending display of the real endoscopic image and the virtual endoscopic image.

14. The diagnostic endoscopic imaging support apparatus as claimed in claim 1, wherein the endoscope comprises a bronchoscope, a large intestine endoscope, a small intestine endoscope, or a capsule endoscope.

15. The diagnostic endoscopic imaging support apparatus as claimed in claim 1, wherein the tubular tissue comprises a bronchus, a large intestine, or a small intestine.

16. The diagnostic endoscopic imaging support apparatus as claimed in claim 1, wherein the three-dimensional image data comprise Computed Tomography (CT) image data.

17. The diagnostic endoscopic imaging support apparatus as claimed in claim 1, wherein the three-dimensional image data comprise Magnetic Resonance (MR) image data.

18. A diagnostic endoscopic imaging support method, comprising:

by a processor,

obtaining three-dimensional image data of a subject;

extracting and obtaining tubular tissue shape data representing a shape of a tubular tissue in the subject from the three-dimensional image data;

obtaining endoscope route data representing a series of positions along a route of an endoscope inserted into the subject and including a position of a tip of the endoscope,

wherein the position of the tip of the endoscope corresponds to a front-most position along said series of positions;

extracting, from the tubular tissue shape data and the endoscope route data, target points at which curvature of the shape of the tubular tissue indicated by the tubular tissue shape data and curvature of the route of the endoscope indicated by the endoscope route data commonly exceed a predetermined threshold;

using the extracted target points, performing matching between the shape of the tubular tissue represented by the tubular tissue shape data and the series of positions along the route of the endoscope including the position of the tip of the endoscope represented by the endoscope route data,

wherein, when the route of the endoscope in the subject is changed, a detection of a variation in the route of the endoscope is made and the matching is performed in real-time for the part of the route of the endoscope for which the variation has been detected, and reuses the result of previous matching for the remaining part of the route without performing the matching anew; and

transforming the series of positions along the route of the endoscopic based on a result of the matching; and

displaying the transformed series of positions along the route of the endoscope in the subject in an image obtained from the three-dimensional image data.

19. A non-transitory computer readable medium on which is recorded a diagnostic endoscopic imaging support program for causing a computer to execute a process, the process comprising:

obtaining three-dimensional image data of a subject;

extracting and obtaining tubular tissue shape data representing a shape of a tubular tissue in the subject from the three-dimensional image data;

obtaining endoscope route data representing a set of positions alone a route of an endoscope inserted into the subject and including a position of a tip of the endoscope,

wherein the position of the tip of the endoscope corresponds to a front-most position along said series of positions;

extracting, from the tubular tissue shape data and the endoscope route data, target points at which curvature of the shape of the tubular tissue indicated by the tubular tissue shape data and curvature of the route of the endoscope indicated by the endoscope route data commonly exceed a predetermined threshold;

using the extracted target points, performing matching between the shape of the tubular tissue represented by the tubular tissue shape data and the series of positions along the route of the endoscope including the position of the tip of the endoscope represented by the endoscope route data,

wherein, when the route of the endoscope in the subject is changed, a detection of a variation in the route of the endoscope is made and the matching is performed in real-time for the part of the route of the endoscope for which the variation has been detected, and reuses the result of previous matching for the remaining part of the route without performing the matching anew; and

transforming the series of positions along the route of the endoscope based on a result of the matching; and

controlling a display to display the transformed series of positions along the route of the endoscope in the subject in an image obtained from the three-dimensional image data.