IMAGE PROCESSING DEVICE, IMAGE PROCESSING PROGRAM AND IMAGE PROCESSING METHOD

• US 20100061597A1
• Filed: 11/18/2009
• Published: 03/11/2010
• Est. Priority Date: 06/05/2007
• Status: Active Grant

• Current Assignee: Olympus Corporation
• Original Assignee: Olympus Corporation
• Inventors: KANDA, Yamato, MATSUZAKI, Hiroshi
• Granted Patent: US 8,107,704 B2
• Time in Patent Office: Days
• US Class Current: 382/107
• CPC Class Codes: A61B 1/041 Capsule endoscopes for imaging
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/30028 Colon; Small intestine
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: Y10S 128/922 including image analysis

Abstract

An image processing device for processing time-series intraluminal images captured by an imaging device moving in an in-vivo lumen. The image processing device includes a structural area extracting unit that extracts a structural area from the intraluminal image; a corresponding area extracting unit that extracts a corresponding area corresponding to the structural area from the intraluminal image at a second time point different from a first time point at which the structural area is extracted; a lumen deep portion extracting unit that extracts a lumen deep portion, on which a deep portion in the lumen is captured, from the intraluminal image; and a movement amount estimating unit that estimates a movement amount of the imaging device based on positions of the structural area, the corresponding area and the lumen deep portion.

Claims

1. An image processing device for processing time-series intraluminal images captured by an imaging device moving in an in-vivo lumen, the image processing device comprising:

a structural area extracting unit that extracts a structural area from the intraluminal image;

a corresponding area extracting unit that extracts a corresponding area corresponding to the structural area from the intraluminal image at a second time point different from a first time point at which the structural area is extracted;

a lumen deep portion extracting unit that extracts a lumen deep portion, on which a deep portion in the lumen is captured, from the intraluminal image; and

a movement amount estimating unit that estimates a movement amount of the imaging device based on positions of the structural area, the corresponding area and the lumen deep portion.

2. The image processing device according to claim 1, wherein the movement amount estimating unit obtains a first projected distance by projecting a first distance from the imaging device to the structural area at the first time point on an intraluminal wall surface using the positions of the structural area, the corresponding area and the lumen deep portion, and obtains a second projected distance by projecting a second distance from the imaging device to the corresponding area at the second time point on the intraluminal wall surface, and further calculates a difference between the first projected distance and the second projected distance to estimate the movement amount of the imaging device.

3. The image processing device according to claim 2, wherein the movement amount estimating unit calculates a first angle, which is an angle between a vector from an optical center corresponding to a main point of an optical system of the imaging device to the lumen deep portion and a vector from the optical center to the structural area at the first time point, to obtain the first projected distance using the calculated first angle and a radius of the lumen, and calculates a second angle, which is an angle between the vector from the optical center corresponding to the main point of the optical system of the imaging device to the lumen deep portion and the vector from the optical center to the corresponding area at the second time point, to obtain the second projected distance using the calculated second angle and the radius of the lumen.

4. The image processing device according to any one of claims 1 to 3, wherein the structural area extracting unit extracts an area, in which a mucosal membrane in the lumen is captured, as the structural area.

5. The image processing device according to any one of claims 1 to 4, wherein the structural area extracting unit extracts an area, in which at least one of wrinkles of the mucosal membrane in the lumen and blood vessels on a surface of the mucosal membrane in the lumen are captured, as the structural area.

6. The image processing device according to any one of claims 1 to 5, wherein the structural area extracting unit extracts an area having a distribution area of pixel values larger than a predetermined threshold as the structural area.

7. The image processing device according to any one of claims 1 to 6, wherein the structural area extracting unit extracts a plurality of areas in the intraluminal image as the structural areas.

8. The image processing device according to any one of claims 1 to 7, wherein the corresponding area extracting unit performs a matching process by setting the structural area as a template on the in-vivo intraluminal images at different time points and extracts an area having similarity at the time of the matching higher than a predetermined threshold as the corresponding area.

9. The image processing device according to any one of claims 1 to 8, wherein the lumen deep portion extracting unit extracts an area, in which dark pixels are gathered in the intraluminal image, as the lumen deep portion.

10. The image processing device according to any one of claims 1 to 9, further comprising a position estimating unit that estimates a position of the imaging device in the lumen at each time when capturing each intraluminal image by accumulating a plurality of values of the movement amount estimated by processing a plurality of the intraluminal images.

11. An image processing device, comprising:

an imaging device movement amount estimating unit that processes time-series intraluminal images captured by an imaging device moving in an in-vivo lumen and estimates a movement amount of the imaging device.

12. The image processing device according to claim 11, wherein the imaging device movement amount estimating unit comprises:

a predetermined area extracting unit that extracts predetermined areas in the lumen in the intraluminal images at different time points; and

a movement amount estimating unit that estimates the movement amount of the imaging device based on positions of the predetermined areas.

13. The image processing device according to claim 12, wherein the predetermined areas are structural areas extracted from the intraluminal images, and

the movement amount estimating unit calculates the movement amount of the imaging device estimated based on positions of the structural areas for each of the structural areas, and sets an average value of a plurality of movement amounts calculated for each of the structural areas as the movement amount of the imaging device.

14. The image processing device according to claim 12 or 13, wherein the imaging device starts moving from an entrance of the in-vivo lumen to reach an exit of the lumen,

the movement amount estimating unit comprises:

a total sum value calculating unit that obtains a total sum value that is a total sum of the movement amounts of the imaging device until the imaging device reaches the exit based on the movement amounts of the imaging device estimated for each predetermined time interval; and

an accumulated value calculating unit that obtains an accumulated value obtained by integrating the movement amounts of the imaging device from start of movement of the imaging device to predetermined time based on the movement amounts of the imaging device estimated for each predetermined time interval, and

the movement amount estimating unit calculates a relative position of the imaging device in the lumen at the predetermined time based on the total sum value and the accumulated value.

15. The image processing device according to claim 14, wherein the movement amount estimating unit calculates the relative position of the imaging device in the lumen at the predetermined time by dividing the accumulated value by the total sum value.

16. An image processing program that, when executed by a computer for processing time-series intraluminal images captured by an imaging device moving in an in-vivo lumen, causes the computer to perform:

a structural area extracting process that extracts a structural area from the intraluminal image;

a corresponding area extracting process that extracts a corresponding area corresponding to the structural area from the intraluminal image at a second time point different from a first time point at which the structural area is extracted;

a lumen deep portion extracting process that extracts a lumen deep portion, on which a deep portion in the lumen is captured, from the intraluminal image; and

a movement amount estimating process that estimates a movement amount of the imaging device based on positions of the structural area, the corresponding area and the lumen deep portion.

17. An image processing program that, when executed by a computer, causes the computer to perform:

an imaging device movement amount estimating process that processes time-series intraluminal images captured by an imaging device moving in an in-vivo lumen to estimate a movement amount of the imaging device; and

a position estimating process that accumulates a plurality of values of the movement amount estimated by processing a plurality of the intraluminal images by the imaging device movement amount estimating process, thereby estimating a position of the imaging device in the lumen at each time when capturing each intraluminal image.

18. An image processing method for processing time-series intraluminal images captured by an imaging device moving in an in-vivo lumen, the method comprising:

a structural area extracting step that extracts a structural area from the intraluminal image;

a corresponding area extracting step that extracts a corresponding area corresponding to the structural area from the intraluminal image at a second time point different from a first time point at which the structural area is extracted;

a lumen deep portion extracting step that extracts a lumen deep portion, on which a deep portion in the lumen is captured, from the intraluminal image; and

a movement amount estimating step that estimates a movement amount of the imaging device based on positions of the structural area, the corresponding area and the lumen deep portion.

19. An image processing method, comprising:

an imaging device movement amount estimating step that processes time-series intraluminal images captured by an imaging device moving in an in-vivo lumen to estimate a movement amount of the imaging device; and

a position estimating step that accumulates a plurality of values of the movement amount estimated by processing a plurality of the intraluminal images by the imaging device movement amount estimating step, thereby estimating a position of the imaging device in the lumen at each time when capturing each intraluminal image.

20. An image processing device for analyzing images of a row of images captured in time series by a moving imaging device, the image processing device comprising:

a movement amount calculating unit that calculates a positional displacement of an object on an image area using correlation between images, and calculates a movement amount the imaging device moves between captures of the images based on the positional displacement; and

a distance calculating unit that calculates a movement distance of the imaging device by accumulating and adding the movement amounts.

21. The image processing device according to claim 20, further comprising a comparison area setting unit that sets a comparison area on the image area of each image, wherein

the movement amount calculating unit calculates the movement amount based on the positional displacement of the comparison area in the image area.

22. The image processing device according to claim 21, wherein the movement amount calculating unit calculates the movement amount when the positional displacement amount on the image area of the comparison area between the images satisfies a condition defined by a predetermined threshold.

23. The image processing device according to any one of claims 20 to 22, wherein the distance calculating unit accumulates and adds the movement amount satisfying the condition defined by the predetermined threshold out of the movement amounts calculated by the movement amount calculating unit.

24. The image processing device according to any one of claims 20 to 23, wherein the movement amount calculating unit calculates the positional displacement in a plurality of areas on the image area, calculates an area movement amount for each of the plurality of areas, and applies a statistical process to the area movement amounts, thereby calculating the movement amount.

25. The image processing device according to any one of claims 21 to 24, further comprising a limiting unit that limits an area, in which the comparison area is set;

wherein

the comparison area setting unit sets the comparison area within the range limited by the limiting unit.

26. The image processing device according to any one of claims 20 to 25, further comprising a specifying unit that specifies an observation target captured in the image, wherein

the distance calculating unit accumulates and adds the movement amounts for each of the observation targets to calculate the movement distance of the imaging device.

27. The image processing device according to any one of claims 20 to 26, further comprising a specifying unit that specifies an observation target captured in the image, wherein

the movement amount calculating unit calculates the movement amount using a calculation condition depending on the observation target specified by the specifying unit.

28. The image processing device according to any one of claims 20 to 27, further comprising a capturing distance calculating unit that calculates a capturing distance between an area of interest captured in the image and the imaging device, wherein

the distance calculating unit calculates a distance from a predetermined position to a position corresponding to the area of interest as an accumulated movement distance the imaging device moves, by adding a value obtained by accumulating and adding the movement amounts and a value of the capturing distance.

29. The image processing device according to claim 28, further comprising a display unit that displays the images of the row of images;

and

an input unit that inputs the area of interest on the image displayed by the display unit.

30. The image processing device according to any one of claims 20 to 29, wherein the row of images is a row of intra-gastrointestinal tract images captured by a capsule endoscope inserted into a gastrointestinal tract of a subject.

31. An image processing program that, when executed by a computer for analyzing images of a row of images captured in time series by a moving imaging device, causes the computer to perform:

a movement amount calculating process that calculates a positional displacement of an object on an image area using correlation between images, and calculates a movement amount the imaging device moves between captures of the images based on the positional displacement; and

a distance calculating process that calculates a movement distance of the imaging device by accumulating and adding the movement amounts.

32. An image processing method for analyzing images of a row of images captured in time series by a moving imaging device, the method comprising:

a movement amount calculating step that calculates a positional displacement of an object on an image area using correlation between images, and calculates a movement amount the imaging device moves between captures of the images based on the positional displacement; and

a distance calculating step that calculates a movement distance of the imaging device by accumulating and adding the movement amounts.