Endoscopic image processing device, method and program

• US 8,939,892 B2
• Filed: 08/30/2011
• Issued: 01/27/2015
• Est. Priority Date: 08/31/2010
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Miyamoto, Masaki
• Primary Examiner(s): Nguyen, Anhtuan T.
• Assistant Examiner(s): NEWTON, ALEXANDRA L
• Application Number:
  US13/221,405
• Publication Number:
  US 20120053408A1
• Time in Patent Office: 1,246 Days
• Field of Search: 600/109, 600/114, 600/117, 600/118, 600/416, 600/417
• US Class Current: 600/109
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/10072 Tomographic images
• CPC Class Codes: G06T 2207/10124 Digitally reconstructed radiograph [DRR]
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G06T 2219/2012 Colour editing, changing, or manipulating; Use of colour codes
• CPC Class Codes: G06T 7/74 involving reference images or patches

Abstract

Endoscopic images and virtual endoscopic images are obtained. Then, an endoscopic image captured at a predetermined position of the anatomical structure is extracted from the obtained endoscopic images, and a comparative virtual endoscopic image virtually generated as if it is captured at a position corresponding to the predetermined position is extracted from the obtained virtual endoscopic images, and the extracted images are associated with each other. A three-dimensional position corresponding to each pixel forming the endoscopic image captured at the predetermined position is calculated based on a three-dimensional position of each pixel forming the comparative virtual endoscopic image. Then, volume data is generated from the endoscopic image captured at the predetermined position based on a pixel value of each pixel forming the endoscopic image and the three-dimensional position calculated for each pixel.

Claims

1. An endoscopic image processing device comprising:

a memory storing a plurality of components; and

a processor configured to execute the plurality of components, which comprise;

an endoscopic image interface configured to obtain a plurality of endoscopic images captured with an endoscope inserted into an anatomical structure of a subject;

a volume data interface configured to obtain volume data of the anatomical structure acquired through tomographic imaging wherein the obtained volume data is three dimensional image data;

a virtual endoscopic image generator configured to generate a plurality of virtual endoscopic images of the anatomical structure from the obtained volume data;

an associator configured to extract an endoscopic image captured at a predetermined position of the anatomical structure from the obtained endoscopic images, further configured to extract a comparative virtual endoscopic image virtually generated as if being captured at a position corresponding to the predetermined position from the obtained virtual endoscopic images, and further configured to associate the extracted endoscopic image and the comparative virtual endoscopic image with each other;

a position calculator configured to calculate a three-dimensional position of each pixel forming the anatomical structure based on the obtained volume data, specifying each pixel forming the associated comparative virtual endoscopic image corresponding to each pixel forming the endoscopic image captured at the predetermined position, specifying each pixel on an inner wall of the anatomical structure in the obtained volume data corresponding to each pixel forming the comparative virtual endoscopic image, and further configured to calculate, as a three-dimensional position corresponding to each pixel forming the endoscopic image, the three-dimensional position of the pixel on the inner wall corresponding to each pixel forming the associated comparative virtual endoscopic image corresponding to each pixel forming the endoscopic image; and

an endoscopic volume data generator configured to generate volume data, which includes the pixels forming the anatomical structure and the pixels positioned in the calculated three-dimensional positions corresponding to each pixel forming the endoscopic image and further configured to generate an endoscopic volume data by inserting a pixel value of each pixel forming the endoscopic image as the pixel value of each pixel positioned in the calculated three-dimensional positions corresponding to each pixels forming the endoscopic image based on the generated volume data and the endoscopic image,

wherein the endoscopic volume data is three dimensional image data, and

wherein the endoscopic volume data generator is configured to insert a pixel value of a certain pixel among the pixels forming the endoscopic image as pixel values of a plurality of pixels included in a predetermined regions which includes the three-dimensional positions of the endoscopic volume data corresponding to the certain pixel.

2. The endoscopic image processing device as claimed in claim 1, wherein the components executed by the processor further comprise:

a reconstructed endoscopic image generator configured to generate a reconstructed endoscopic image reconstructed from the endoscopic volume data generated from the endoscopic image; and

a display controller configured to cause the generated reconstructed endoscopic image to be displayed on a display.

3. The endoscopic image processing device as claimed in claim 1, wherein the endoscopic volume data generator configured to insert the pixel value of the certain pixel as pixel values of a plurality of pixels included in the predetermined region which has a spherical shape region having a predetermined radius.

4. The endoscopic image processing device as claimed in claim 1, wherein the endoscopic images obtained by the endoscopic image obtaining interface are spectral images.

5. The endoscopic image processing device as claimed in claim 2, wherein the generated reconstructed endoscopic image is a stretched development image or a stretched cross-sectional image of the anatomical structure.

6. The endoscopic image processing device as claimed in claim 2, further comprising unassociated area extracting means for extracting, from the endoscopic volume data, an unassociated area formed by three-dimensional positions that are associated with none of the pixels forming the endoscopic image captured at the predetermined position.

7. The endoscopic image processing device as claimed in claim 6, further comprising

medical image generating means for generating a medical image from the volume data,

wherein the display control means causes a marking indicating the unassociated area to be displayed on the medical image generated from the volume data.

8. The endoscopic image processing device as claimed in claim 6, wherein the display control means causes a marking indicating the unassociated area to be displayed on the reconstructed endoscopic image generated from the endoscopic volume data.