Control method, information terminal, recording medium, and determination method

• US 10,366,785 B2
• Filed: 12/13/2017
• Issued: 07/30/2019
• Est. Priority Date: 12/22/2016
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Inventors: Wakasugi, Kensuke, Kondo, Kenji, Takata, Kazutoyo, Kimura, Hirohiko, Sakai, Toyohiko
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US15/839,919
• Publication Number:
  US 20180182481A1
• Time in Patent Office: 594 Days
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/10081 Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 2207/30061 Lung
• CPC Class Codes: G06T 2207/30096 Tumor; Lesion
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06V 10/235 based on user input or interaction
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/267 by performing operations on regions, e.g. growing, shrinking or watersheds
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/464 using a plurality of salient features, e.g. bag-of-words [BoW] representations
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 2201/03 : Recognition of patterns in medical or anatomical images
• CPC Class Codes: G16H 30/40 : for processing medical images, e.g. editing
• CPC Class Codes: G16H 40/63 : for local operation
• CPC Class Codes: G16H 50/20 : for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/70 : for mining of medical data, e.g. analysing previous cases of other patients

Abstract

If a lesion included in a specification target image is a texture lesion, a probability image calculation unit calculates a probability value indicating a probability that each of a plurality of pixels of the specification target image is included in a lesion area. An output unit calculates, as a candidate area, an area including pixels whose probability values are equal to or larger than a first threshold in a probability image obtained from the probability image calculation unit and, as a modification area, an area including pixels whose probability values are within a certain probability range including the first threshold. An input unit detects an input from a user on a pixel in the modification area. A lesion area specification unit specifies a lesion area on the basis of the probability image, the candidate area, the modification area, and user operation information.

Claims

1. A method for controlling an information terminal including a display, the information terminal being connected to a case retrieval system that retrieves a medical image by referring to a medical image database, the method comprising:

(a) receiving a first medical image including a plurality of pixels;

(b) calculating, if a lesion included in the first medical image is a texture lesion, a probability value indicating a probability that each of the plurality of pixels of the first medical image is included in a lesion area by inputting a pixel value of the pixel and pixel values of surrounding pixels of the pixel to an identification device for identifying a predetermined lesion area indicating the texture lesion, the texture lesion being a lesion including one of a plurality of particular shadow patterns;

(c) displaying, on the display, a second medical image obtained by superimposing a candidate area and a modification area of the candidate area upon the first medical image, the candidate area being determined on the basis of a pixel whose probability value is larger than a first threshold among the plurality of pixels, the modification area being determined on the basis of a pixel whose probability value is within a probability range among the plurality of pixels, the probability range including the first threshold;

(d) detecting an input from a user on the second medical image displayed on the display; and

(e) displaying a third medical image obtained by superimposing the lesion area upon the first medical image, the lesion area being determined by modifying the candidate area on the basis of the input from the user and the modification area.

2. The method according to claim 1,

wherein, in (c), the modification area to be superimposed upon the second medical image is divided into a plurality of sub-areas, and

wherein, in each of the plurality of sub-areas, a longest straight line connecting two pixels selected from the sub-area is equal to or shorter than a second threshold.

3. The method according to claim 2,

wherein, in (d), an input from the user as to whether to add or exclude each of the plurality of sub-areas to or from the candidate area is detected, and

wherein, in (e), a candidate area determined on the basis of the input from the user is displayed as the lesion area.

4. The method according to claim 1,

wherein, in (c), the modification area is displayed such that the modification area includes the candidate area,

wherein, in (d), an input from the user on a pixel included in the modification area is detected, and

wherein, in (e), the lesion area is an area obtained by enlarging or reducing the candidate area on the basis of a part of the modification area in accordance with the input from the user.

5. The method according to claim 4,

wherein, in (e), a degree to which the candidate area is enlarged or reduced when an input from the user on a first pixel has been detected is set lower than when an input from the user on a second pixel has been detected, and

wherein an inclination of probability values of surrounding pixels of the first pixel is larger than an inclination of probability values of surrounding pixels of the second pixel.

6. The method according to claim 5,

wherein, in (d), a drag operation performed by the user is detected as the input, and

wherein, in (e), the first and second pixels are each a pixel at a start point of the drag operation, and the degree to which the candidate area is enlarged or reduced is proportional to an amount of movement in the drag operation and inversely proportional to the inclination.

7. An information terminal including a display, the information terminal being connected to a case retrieval system that retrieves a medical image by referring to a medical image database, the information terminal comprising:

a probability image calculator that receives a first medical image including a plurality of pixels and, if a lesion included in the first medical image is a texture lesion including one of a plurality of particular shadow patterns, calculates a probability value included in a lesion area by inputting a pixel value of each of the plurality of pixels of the first medical image and pixel values of surrounding pixels of the pixel to an identification device for identifying a predetermined lesion area;

an outputter that determines, for the first medical image, a candidate area on the basis of a pixel whose probability value is equal to or larger than a first threshold among the plurality of pixels and a modification area of the candidate area on the basis of a pixel whose probability value is within a probability range including the first threshold among the plurality of pixels, superimposes the candidate area and the modification area upon the first medical image, and displays a second medical image on the display;

an inputter that detects an input from a user on the second medical image displayed on the display; and

a lesion area specifier that specifies a lesion area by modifying the candidate area on the basis of the input from the user detected by the inputter and the modification area,

wherein the outputter displays, on the display, a third medical image upon which the lesion area is superimposed.

8. The information terminal according to claim 7,

wherein the probability image calculator divides the modification area to be superimposed upon the second medical image into a plurality of sub-areas, and

wherein, in each of the plurality of sub-areas, a longest straight line connecting two pixels selected from the sub-area is equal to or shorter than a second threshold.

9. The information terminal according to claim 8,

wherein the inputter detects an input from the user as to whether to add or exclude each of the plurality of sub-areas to or from the candidate area, and

wherein the lesion area specifier displays a candidate area determined on the basis of the input from the user as the lesion area.

10. The information terminal according to claim 9,

wherein the outputter displays the modification area such that the modification area includes the candidate area,

wherein the inputter detects an input from the user on a pixel included in the modification area, and

wherein the lesion area specifier determines the lesion area by enlarging or reducing the candidate area on the basis of a part of the modification area in accordance with the input from the user.

11. The information terminal according to claim 10,

wherein the lesion area specifier sets a degree to which the candidate area is enlarged or reduced when an input from the user on a first pixel has been detected lower than when an input from the user on a second pixel has been detected, and

wherein an inclination of probability values of surrounding pixels of the first pixel is larger than an inclination of probability values of surrounding pixels of the second pixel.

12. The information terminal according to claim 11,

wherein the inputter detects a drag operation performed by the user as the input, and

wherein the first and second pixels are each a pixel at a start point of the drag operation, and the degree to which the candidate area is enlarged or reduced is proportional to an amount of movement in the drag operation and inversely proportional to the inclination.

13. A recording medium storing a control program for causing a device including a processor connected to a case retrieval system that retrieves a medical image by referring to a medical image database to perform a process, the process comprising:

receiving a first medical image including a plurality of pixels;

calculating, if a lesion included in the first medical image is a texture lesion including one of a plurality of particular shadow patterns, a probability value indicating a probability that each of the plurality of pixels of the first medical image is included in a lesion area by inputting a pixel value of the pixel and pixel values of surrounding pixels of the pixel to an identification device for identifying a predetermined lesion area;

determining, for the first medical image, a candidate area on the basis of a pixel whose probability value is equal to or larger than a first threshold among the plurality of pixels and a modification area of the candidate area on the basis of a pixel whose probability value is within a probability range including the first threshold among the plurality of pixels, superimposing the candidate area and the modification area upon the first medical image, and displaying a second medical image on a display;

detecting an input from a user on the second medical image displayed on the display;

specifying a lesion area by modifying the candidate area on the basis of the detected input from the user and the modification area; and

displaying, on the display, a third medical image upon which the lesion area is superimposed.

14. A determination method comprising:

receiving a medical image including pixels, each of the pixels having a pixel value;

calculating probability values each corresponding to the pixels, each of the probability values indicating a probability that a corresponding pixel is included in a lesion area;

determining, from among a first area, a second area, a third area, and a fourth area, an area to which each of the pixels belongs, each of pixels included in the first area having a pixel value equal to or larger than an upper limit, each of pixels included in the second area having a pixel value equal to or larger than a first threshold but smaller than the upper limit, each of pixels included in the third area having a pixel value larger than a lower limit but smaller than the first threshold, each of pixels included in the fourth area having a pixel value equal to or smaller than the lower limit, the upper limit being larger than the first threshold, the first threshold being larger than the lower limit;

receiving first selections of one or more areas included in the second area;

determining one or more fifth areas being included in the second area and not being selected in the first selections;

receiving second selections of one or more areas included in the third area;

determining one or more sixth areas being included in the third area and being selected in the second selections; and

determining pixels included in the first area, the one or more fifth areas, and the one or more sixth areas are included in the lesion area.