Image processing device, computer-readable recording device, and image processing method

• US 9,186,051 B2
• Filed: 01/17/2012
• Issued: 11/17/2015
• Est. Priority Date: 07/23/2009
• Status: Active Grant

• Current Assignee: Olympus Corporation
• Original Assignee: Olympus Corporation
• Inventors: Hirota, Masashi
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): MISTRY, ONEAL R
• Application Number:
  US13/351,342
• Publication Number:
  US 20120114203A1
• Time in Patent Office: 1,400 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A61B 1/041 Capsule endoscopes for imaging
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/30028 Colon; Small intestine
• CPC Class Codes: G06T 2207/30092 Stomach; Gastric
• CPC Class Codes: G06T 7/0016 involving temporal comparison
• CPC Class Codes: G06V 10/267 by performing operations on regions, e.g. growing, shrinking or watersheds
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]

Abstract

An image processing device includes: an interest area detector that detects interest areas included in a time-series image group captured in time series; a calculation processing unit that calculates feature amounts indicative of features of the interest areas; an area classification unit that classifies the interest areas into area groups, based on the feature amounts of the interest areas and time-series positions of time-series images including the interest areas; a group feature amount calculation unit that calculates a group feature amount indicative of a feature of each of the area groups; an area selection unit that selects one or more representative areas of the interest areas belonging to the area groups, from among the area groups; and a representative image output unit that outputs one or more representative images including the representative areas in the time-series image group.

Claims

1. An image processing device, comprising:

an interest area detector that detects interest areas included in time-series images captured in time series;

a calculation processing unit that calculates first feature amounts indicative of features of the interest areas;

an area classification unit that classifies the interest areas into area groups, based on the first feature amounts of the interest areas and the position in time for each of the captured time-series images that includes the interest areas;

an area selection unit that selects, based on second feature amounts of the interest areas in each of the area groups, one or more representative areas representing the interest areas belonging to each of the area groups, from among the interest areas belonging to each of the area groups; and

a representative image output unit that outputs one or more representative images including the representative areas in the time-series images.

2. The image processing device according to claim 1, wherein the area classification unit classifies the interest areas into area groups, based on the degree of similarity indicative of similarity in the first feature amount between the detected interest areas and a relationship in time-series position between the time-series images including the detected interest areas.

3. The image processing device according to claim 2, wherein the area classification unit includes:

an adjacent state determination unit that determines whether the interest areas detected by the interest area detector are adjacent to each other in time series; and

a degree-of-similarity determination unit that determines a degree of similarity in the first feature amount between the interest areas calculated by the calculation processing unit,

wherein the area classification unit classifies the interest areas into area groups, based on adjacent states of the plurality of interest areas determined by the adjacent state determination unit and the degree of similarity in the first feature amount in the interest areas determined by the degree-of-similarity determination unit.

4. The image processing device according to claim 3, wherein

the adjacent state determination unit determines whether the interest areas in each of area groups selected from the plurality of interest groups are adjacent or identical to each other in time series, and

if the interest areas in each of the area groups are adjacent or identical to each other in time series, the area classification unit integrates the plurality of area groups adjacent or identical to each other in time series into one area group.

5. The image processing device according to claim 1, wherein the first feature amounts of the interest areas are color feature amounts.

6. The image processing device according to claim 5, wherein

the calculation processing unit includes:

a pixel value converter that converts values of pixels belonging to the interest areas to values of L*a*b space; and

an average calculation unit that calculates averages of values of L*a*b space converted and output by the pixel value converter, as color feature amounts of the interest areas.

7. The image processing device according to claim 1, further comprising a group feature amount calculation unit that calculates a group feature amount indicative of a feature of each of the area groups based on the second feature amounts of the interest areas in each of the area groups,

wherein the group feature amount calculation unit calculates dispersion of the second feature amounts of the interest areas in a feature space formed by a coordinate axis of the second feature amounts of the interest areas and a time-series coordinate axis, and calculates the group feature amount of each of the area groups based on the calculated dispersion.

8. The image processing device according to claim 1, further comprising a group feature amount calculation unit that calculates a group feature amount indicative of a feature of each of the area groups based on the second feature amounts of the interest areas in each of the area groups,

wherein the area selection unit includes a number-of-selection decision unit that decides a number of selection(s) of the representative area(s) based on the group feature amount, and selects a number of the representative area(s) equal to the number of selection(s), from the area groups.

9. The image processing device according to claim 8, wherein the number-of-selection decision unit includes a function unit that sets a function indicative of a relationship between the group feature amounts and the number of selection(s), and decides the number of selection(s) based on the function set by the function unit.

10. The image processing device according to claim 8, wherein the area selection unit includes a sub-classification processing unit that sub-classifies the plurality of interest areas into a number of similarity group(s) equal to the number of selection(s), based on the second feature amounts of the plurality of interest areas included in the area groups, and selects one interest area for each of the similarity groups to select a number of the representative area(s) equal to the number of selection(s).

11. The image processing device according to claim 10,

wherein the area selection unit includes:

a barycenter calculation unit that calculates barycenters of the second feature amounts of the plurality of interest areas included in the similarity groups for each of the similarity groups; and

a closest area selection unit that selects interest areas closest to the barycenters, from the plurality of interest areas included in the similarity groups, and

wherein the area selection unit selects the closest interest areas as the representative areas from each of the similarity groups.

12. The image processing device according to claim 8, wherein the area selection unit calculates time-series coordinates that divide distribution of the second feature amounts of the plurality of interest areas at equal distances in a time-series direction, in correspondence with the number of selection(s), and selects a number of the interest area(s) closest to the time-series coordinate(s) equal to the number of selection(s), as the representative areas from the plurality of interest areas.

13. The image processing device according to claim 1, wherein the time-series images are in-vivo images of an inside of a digestive tract of a subject captured in time series.

14. The image processing device according to claim 13, wherein the interest areas are lesion areas or mucosal areas in an inside of a body of a subject.

15. The image processing device according to claim 1, further comprising a group feature amount calculation unit that calculates a group feature amount indicative of a feature of each of the area groups based on the second feature amounts of the interest areas in each of the area groups.

16. The image processing device according to claim 1, wherein the first feature amounts are the same as the second feature amounts.

17. The image processing device according to claim 1, wherein the area classification unit classifies the interest areas into the area groups, based on a distribution state of feature amounts of the interest areas and a time-series distribution state of feature amounts of the interest areas.

18. A non-transitory computer-readable recording device with an executable program stored thereon, wherein the program instructs a processor to perform:

detecting interest areas included in time-series images captured in time series;

calculating first feature amounts indicative of features of the interest areas;

classifying the interest areas into area groups, based on the first feature amounts of the interest areas and the position in time for each of the captured time-series images that includes the interest areas;

selecting, based on second feature amounts of the interest areas in each of the area groups, one or more representative areas representing the interest areas belonging to each of the area groups, from among the interest areas belonging to each of the area groups; and

outputting one or more representative images including the representative areas in the time-series Images.

19. The non-transitory computer-readable recording device according to claim 18, wherein the program instructs the processor to perform:

classifying the interest areas into the area groups, based on a distribution state of feature amounts of the interest areas and a time-series distribution state of feature amounts of the interest areas.

20. An image processing method comprising:

detecting interest areas included in time-series images captured in time series;

calculating first feature amounts indicative of features of the interest areas;

classifying the interest areas into area groups, based on the first feature amounts of the interest areas and the position in time for each of the captured time-series images that includes the interest areas;

selecting, based on second feature amounts of the interest areas in each of the area groups, one or more representative areas representing the interest areas belonging to each of the area groups, from among the interest areas belonging to each of the area groups; and

outputting one or more representative images including the representative areas in the time-series images.

21. The image processing method according to claim 20, wherein the step of classifying the interest areas into the area groups comprises:

classifying the interest areas into the area groups, based on a distribution state of feature amounts of the interest areas and a time-series distribution state of feature amounts of the interest areas.