Registered video endoscopy and virtual endoscopy

• US 9,373,166 B2
• Filed: 04/18/2005
• Issued: 06/21/2016
• Est. Priority Date: 04/23/2004
• Status: Active Grant

• Current Assignee: Siemens Healthcare GMBH (Siemens AG)
• Original Assignee: Siemens Medical Solutions USA Incorporated (Siemens AG)
• Inventors: Azar, Fred S.
• Primary Examiner(s): Brutus, Joel F
• Application Number:
  US11/108,440
• Publication Number:
  US 20050251017A1
• Time in Patent Office: 4,082 Days
• Field of Search: 600407-480, 600/173, 606/2
• US Class Current: 1/1
• CPC Class Codes: A61B 1/00158 using magnetic field
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2072 Reference field transducer attached to an instrument or patient
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 5/06 Devices, other than using radiation, for detecting or locating foreign bodies for removing same A61B17/50 ; determining position of probes within or on the body of the patient
• CPC Class Codes: A61B 5/062 using magnetic field
• CPC Class Codes: A61B 90/36 Image-producing devices or illumination devices not otherwise provided for
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/30028 Colon; Small intestine
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

An endo-robot can be tracked and a reconstructed 3D volume of data can be created for virtual endoscopy. The 3D position and orientation of each image captured by the endo-robot is determined. The longitudinal distance traveled inside the structure of interest is determined. The position, orientation and longitudinal distance are used to register the position of the endo-robot to a corresponding position inside the 3D volume of data. Virtual endoscopy can be used to locate areas of interest and correlate clinical findings with the images of the areas of interest.

Claims

1. A computer-implemented method for associating an image with an area of interest in a three-dimensional volume comprising:

modeling a volume as the three-dimensional volume;

tracking an image capturing device within the volume, storing position and orientation information of the image capturing device at a time when at least one image is captured;

determining a longitudinal distance traveled by the image capturing device at the time when the at least one image is captured;

registering the position, orientation and longitudinal distance of the image capturing device to a corresponding position within the three-dimensional volume;

determining an area of interest within the three-dimensional volume; and

determining the at least one image registered to correspond to the area of interest, wherein the at least one image is displayed and the area of interest is characterized.

2. The computer-implemented method of claim 1, wherein registering further comprises:

determining a centerline of the three-dimensional volume;

determining a correspondence between a track of the image capturing device and the centerline of the three-dimensional volume, wherein the image capturing device is an endo-robot; and

registering the at least one image to a position within the three-dimensional volume according to the correspondence.

3. The computer-implemented method of claim 1, wherein registering further comprises determining an initial position of the image capturing device and manually selecting a position corresponding to the initial position in the three-dimensional volume.

4. The computer-implemented method of claim 3, wherein the longitudinal distance is a distance of the image capturing device from the initial position.

5. The computer-implemented method of claim 1, wherein determining the at least one image comprises automatically displaying the at least one image in conjunction with the display of the corresponding position within the three-dimensional volume.

6. A non-transitory computer readable medium embodying instructions executed by a processor to perform method steps for associating an image with an area of interest in a three-dimensional volume, the method steps comprising:

modeling a volume as the three-dimensional volume;

tracking an image capturing device within the volume, storing position and orientation information of the image capturing device at a time when at least one image is captured;

determining a longitudinal distance traveled by the image capturing device at the time when the at least one image is captured;

registering the position, orientation and longitudinal distance of the image capturing device to a corresponding position within the three-dimensional volume;

determining an area of interest within the three-dimensional volume; and

determining the at least one image registered to correspond to the area of interest, wherein the at least one image is displayed.

7. The method of claim 6, wherein registering further comprises:

determining a centerline of the three-dimensional volume;

determining a correspondence between a track of the image capturing device and the centerline of the three-dimensional volume, wherein the image capturing device is an endo-robot; and

registering the at least one image to a position within the three-dimensional volume according to the correspondence.

8. The method of claim 6, wherein registering further comprises determining an initial position of the image capturing device and manually selecting a position corresponding to the initial position in the three-dimensional volume.

9. The method of claim 8, wherein the longitudinal distance is a distance of the image capturing device from the initial position.

10. The method of claim 6, wherein determining the at least one image comprises automatically displaying the at least one image in conjunction with the display of the corresponding position within the three-dimensional volume.