Similar image retrieval device, method of operating similar image retrieval device, and similar image retrieval program

• US 9,720,938 B2
• Filed: 07/24/2015
• Issued: 08/01/2017
• Est. Priority Date: 08/21/2014
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Kanada, Shoji
• Primary Examiner(s): FUJITA, KATRINA R
• Application Number:
  US14/807,892
• Publication Number:
  US 20160055394A1
• Time in Patent Office: 739 Days
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 7/0014 using an image reference approach
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 2201/03 Recognition of patterns in medical or anatomical images

Abstract

A feature amount calculation unit 61 calculates a feature amount corresponding to a pattern of a lesion by analyzing an inspection image. A probability calculation unit calculates a first existence probability which is a probability of the pattern of a lesion existing within the inspection image, using a calculation expression. The calculation expression is created in advance by a statistical method on the basis of a relationship between the feature amount and the presence or absence of the pattern of a lesion within an image for learning which is visually determined by a person. A similarity calculation unit calculates a similarity between the inspection image and a case image on the basis of the first existence probability and a second existence probability which is a probability of the pattern of a lesion existing within the case image which is calculated by the statistical method similarly to the first existence probability.

Claims

1. A similar image retrieval device that retrieves a similar image which is similar to a retrieved image from a plurality of instance images, comprising:

a processor configured to,

calculate a feature amount corresponding to a pattern, registered in advance, by analyzing the retrieved image,

calculate a first existence probability of the pattern existing within the retrieved image as an absolute value between 0 to 1, through a statistical method, on the basis of the feature amount, and

calculate a similarity between the retrieved image and each of the plurality of instance images, on the basis of the first existence probability and a second existence probability of the pattern existing within the instance images which is calculated with respect to the plurality of instance images,

wherein the processor calculates at least one of a coexistence probability of the pattern existing in both the instance image and the retrieved image, and a non-coexistence probability of the pattern independently existing in only one of the instance image and the retrieved image, on the basis of the first existence probability and the second existence probability.

2. The similar image retrieval device according to claim 1, wherein the processor uses a calculation expression for calculating the first existence probability, created in advance by the statistical method, using the feature amount as a variable, on the basis of a relationship between the presence or absence of the pattern within an image for learning visually determined by a person and the feature amount.

3. The similar image retrieval device according to claim 1, wherein the processor calculates the coexistence probability or the non-coexistence probability, as the similarity, and

determines that a similarity between the instance image and the retrieved image becomes higher as the coexistence probability increases, or determines that a similarity between the instance image and the retrieved image becomes lower as the non-coexistence probability increases.

4. The similar image retrieval device according to claim 1, wherein the processor calculates a ratio of the coexistence probability to the non-coexistence probability, as the similarity, and

determines that a similarity between the instance image and the retrieved image becomes higher as the ratio becomes higher.

5. The similar image retrieval device according to claim 1, wherein the pattern has multiple types, and

the processor calculates the coexistence probability or the non-coexistence probability of each of the multiple types of patterns,

calculates a total multiplication of the calculated coexistence probabilities or a logarithmic sum of the calculated coexistence probabilities, or a total multiplication of the calculated non-coexistence probabilities or a logarithmic sum of the calculated non-coexistence probabilities, as the similarity, and

determines that a similarity between the instance image and the retrieved image becomes higher as the total multiplication of the coexistence probabilities or the logarithmic sum of the coexistence probabilities increases, or determines that a similarity between the instance image and the retrieved image becomes lower as the total multiplication of the non-coexistence probabilities or the logarithmic sum of the non-coexistence probabilities increases.

6. The similar image retrieval device according to claim 1, wherein the pattern has multiple types, and

the processor calculates a ratio of the coexistence probability to the non-coexistence probability of each of the multiple types of patterns,

calculates a total multiplication of the calculated ratios or a logarithmic sum of the ratios, as the similarity, and

determines that a similarity between the instance image and the retrieved image becomes higher as the total multiplication of the ratios or the logarithmic sum of the ratios increases.

7. The similar image retrieval device according to claim 1, wherein in case where the first existence probability is set to Px, the second existence probability is set to Py, and the coexistence probability is set to Pxy,

a relation of Pxy=Px×

Py is established.

8. The similar image retrieval device according to claim 1, wherein in case where the first existence probability is set to Px, the second existence probability is set to Py, and the non-coexistence probability is set to NPxy,

a relation of NPxy=Px×

(1−

Py)+(1−

Px)×

Py is established.

9. The similar image retrieval device according to claim 1, wherein in case where the first existence probability is set to Px, the second existence probability is set to Py, a minimum value of Px and Py is set to min(Px, Py), and the coexistence probability is set to Pxy,

a relation of Pxy=Px×

min(Px, Py) is established.

10. The similar image retrieval device according to claim 1, wherein in case where the first existence probability is set to Px, the second existence probability is set to Py, a minimum value of Px and Py is set to min(Px, Py), a maximum value of Px and Py is set to max(Px, Py), and the non-coexistence probability is set to NPxy,

in case where Px>

0.5, a relation of NPxy=Px×

{1-min(Px, Py)}+(1−

Px)×

Py is established, and

in case where Px≦

0.5, a relation of NPxy=Px×

(1−

Py)+(1−

Px)×

max(Px, Py) is established.

11. The similar image retrieval device according to claim 1, wherein the processor compares an exclusion range of the second existence probability with the second existence probability, and excludes the instance image in which the second existence probability is in the exclusion range, from candidates for calculating the similarity in the processor.

12. The similar image retrieval device according to claim 11, wherein the processor calculates the exclusion range from a relational expression between the first existence probability and the second existence probability in which the first existence probability is used as a variable.

13. The similar image retrieval device according to claim 1, wherein the processor calculates the feature amount of a region of a portion of the retrieved image.

14. The similar image retrieval device according to claim 1, wherein the processor further calculates a feature amount of the instance image.

15. The similar image retrieval device according to claim 1, wherein the processor further calculates the second existence probability.

16. The similar image retrieval device according to claim 1, wherein the processor outputs a set of the similar image and the similarity, as a retrieval result of the similar image.

17. The similar image retrieval device according to claim 16, wherein the processor outputs a list obtained by sorting a set of a plurality of the similar images and the similarity in order of the similarity.

18. The similar image retrieval device according to claim 1, wherein the retrieved image is an inspection image which is used for a diagnosis in a patient, the instance image is a case image which is the inspection image used for a diagnosis in the past, and the pattern is a pattern of a lesion indicating a symptom of the patient'"'"'s disease.

19. A method of operating a similar image retrieval device, using a computer, which retrieves a similar image which is similar to a retrieved image from a plurality of instance images, comprising:

a feature amount calculation step of causing a processor to calculate a feature amount corresponding to a pattern, registered in advance, by analyzing the retrieved image;

a probability calculation step of causing the processor to calculate a first existence probability of the pattern existing within the retrieved image as an absolute value between 0 to 1, through a statistical method, on the basis of the feature amount calculated in the feature amount calculation step; and

a similarity calculation step of causing the processor to calculate a similarity between the retrieved image and each of the plurality of instance images, on the basis of the first existence probability calculate in the probability calculation step and a second existence probability of the pattern existing in the instance image which is calculated with respect to the plurality of instance images,

wherein the processor calculates at least one of a coexistence probability of the pattern existing in both the instance image and the retrieved image, and a non-coexistence probability of the pattern independently existing in only one of the instance image and the retrieved image, on the basis of the first existence probability and the second existence probability.

20. A non-transitory computer readable recording medium having a similar image retrieval program recorded thereon, the program for retrieving a similar image which is similar to a retrieved image from a plurality of instance images, the medium causing a computer to execute:

a feature amount calculation function of calculating a feature amount corresponding to a pattern, registered in advance, by analyzing the retrieved image;

a probability calculation function of calculating a first existence probability of the pattern existing within the retrieved image as an absolute value between 0 to 1, through a statistical method, on the basis of the feature amount calculated in the feature amount calculation function; and

a similarity calculation function of calculating a similarity between the retrieved image and each of the plurality of instance images, on the basis of the first existence probability calculated in the probability calculation function and a second existence probability of the pattern existing within the instance image which is calculated with respect to the plurality of instance images,

wherein calculating at least one of a coexistence probability of the pattern existing in both the instance image and the retrieved image, and a non-coexistence probability of the pattern independently existing in only one of the instance image and the retrieved image, on the basis of the first existence probability and the second existence probability.