Visualization of relevance for content-based image retrieval

• US 9,495,388 B2
• Filed: 08/30/2010
• Issued: 11/15/2016
• Est. Priority Date: 09/04/2009
• Status: Expired due to Fees

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips N.V.
• Inventors: Unay, Devrim, Ekin, Ahmet
• Primary Examiner(s): Wong, Leslie
• Application Number:
  US13/392,113
• Publication Number:
  US 20120158717A1
• Time in Patent Office: 2,269 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS

Abstract

A system (100) includes a retrieval unit (110) for retrieving an image from the storage of images, on the basis of the similarity of images from the storage of images to a query image. The similarity is defined by means of a similarity function. A relevance unit (120) computes the relevance of a first portion of the retrieved image to a respective first portion of the query image and of a second portion of the retrieved image to the respective second portion of the query image. A visualization unit (130) visualizes the relevance of the first and second portion of the retrieved image to the respective first and second portion of the query image. The relevance of the first and second portion of the retrieved image to the respective first and second portion of the query image is computed using a first and second relevance function. The computed values of the relevance are visualized, e.g. using a color coding and coloring the first and second portion of each retrieved image. The colored portions are easy to see and analyze. Thus, the system facilitates visualizing and comparing retrieved images with respect to each other as well as with the query image.

Claims

1. A system for assisting diagnoses, the system comprising:

a memory which stores a plurality of reference images which have been standardized to have a common scale and position, the reference images being made up of one of pixels and voxels;

one or more processors programmed to;

receive a query image depicting anatomy of a patient to be diagnosed, the query image being made up of one of pixels and voxels;

standardize the query image to the common scale and position,

divide the query image and the reference images with a common grid to define corresponding portions of the query image and the reference images, each portion including a plurality of at least one of pixels and voxels,

receiving an input selecting a plurality of the corresponding, portions,

compute a degree of similarity of each of the selected corresponding portions of the query image with the selected corresponding portions of the reference images,

based on the computed degrees of similarity, select relevant ones of the reference images,

control a display device to display the query image and the selected relevant ones of the reference images with the selected corresponding portions of the selected relevant ones of the reference images marked to indicate the computed degree of similarity with the selected corresponding portion of the query image.

2. The system as claimed in claim 1, wherein the query image is a brain slice image and the reference images are brain slice images of patients diagnosed with Alzheimer'"'"'s disease such that the displayed query image and the displayed reference images with corresponding portions colorized to indicate similarity are indicative of whether the query image is indicative of Alzheimer'"'"'s disease and a progression of the Alzheimer'"'"'s disease.

3. The system as claimed in claim 1, wherein the one or more processors is further programmed to:

control the display device to color code the corresponding portions of the displayed reference images to indicate the computed degree of similarity with semi-transparent colors.

4. The system as claimed in claim 1, wherein the one or more processors is further programmed to:

segment the query image.

5. The system as claimed in claim 1, further comprising:

a user input device which receives user feedback on the computed relevance of the selected corresponding portions of the displayed reference images to the selected corresponding portions of the query image, and wherein the one or mere processors is further programmed to;

update the computed degree of similarity based on the received user feedback.

6. The system as claimed in claim 1, wherein the one or more processors is programmed to select the reference images to be displayed by:

weighting the computed degrees of similarity of the selected corresponding portions of each of the reference images to the selected corresponding portions of the query image the relative weighting being based on a location of each selected portion in the query image; and

adding the weighted computed degrees of similarity of the corresponding portions of each of the reference images.

7. The system as claimed in claim 1, further including:

a diagnostic imager which generates the query image and the reference images;

wherein the memory stores the reference images in conjunction with medical diagnostic information.

8. A method of retrieving reference images from an image memory, the method comprising, with one or more processor;

(a) receiving a query image, the query image and the reference images have corresponding portions;

(b) selecting a plurality of the corresponding portions of the query image and the reference images;

(c) registering the query image with the reference images;

(d) retrieving one of the reference images from the image memory;

(e) comparing the selected corresponding portions of the query image and the one of the reference images using a similarity function to determine a degree of similarity score of each of the corresponding portions;

(f) weighting the degree of similarity scores based on positions of the selected corresponding portions such that similarity of at least one of the selected corresponding portions is weighted more heavily than at least one other of the selected corresponding portions;

(g) combining the weighted degree of similarity scores to generate a relevance value for the retrieved one of the reference images such that the relevance values are positionally weighted;

(h) repeating steps (d)-(g) for a plurality of the reference images;

(i) ranking the reference images based on the relevance values;

(j) selecting a plural number of the reference images based on the ranking;

(k) controlling a display device to display the query image and the selected plural number of reference images with the corresponding portions of the query image of each of displayed reference images denoted, such that as viewer can compare the corresponding portions of the query image and the displayed reference images.

9. A non-transitory computer readable medium carrying a computer program for controlling one or more processors to perform the method as claimed in claim 8.

10. The method as claimed in claim 8, further including, with the one or more processors:

controlling the display device to indicate the degree of similarity score of each of the selected portions of each of the displayed reference image using semi-transparent color coding.

11. The method as claimed in claim 8, further including:

with the one or more processors, segmenting the query image and the reference images.

12. The method as claimed in claim 10, further including, with the one or more processors:

standardizing the reference images and the query image to a common size and position;

dividing each standardized reference image and each query image with a common grid to define the plurality of corresponding portions.

13. The method as claimed in claim 10, further including:

with a user input device, inputting feedback on the similarity of the corresponding portions of the displayed query image and the displayed reference images.

14. The method as claimed in claim 8, further including:

wherein the reference images are anatomical diagnostic brain images of patients diagnosed with Alzheimer'"'"'s disease and the query image is a brain image of a patient.