Method and apparatus for annotating medical image

• US 10,755,411 B2
• Filed: 07/31/2018
• Issued: 08/25/2020
• Est. Priority Date: 09/27/2017
• Status: Active Grant

• Current Assignee: Baidu Online Network Technology (Beijing) Co., Ltd (Baidu Incorporated)
• Original Assignee: Baidu Online Network Technology (Beijing) Co., Ltd (Baidu Incorporated)
• Inventors: Zhang, Shaoting, Zhang, Weidong, Duan, Qi
• Primary Examiner(s): Miller, Brandon J
• Application Number:
  US16/051,214
• Publication Number:
  US 20190096060A1
• Time in Patent Office: 756 Days
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: G06T 2207/10081 Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/10088 Magnetic resonance imaging [MRI]
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30041 Eye; Retina; Ophthalmic
• CPC Class Codes: G06T 2207/30061 Lung
• CPC Class Codes: G06T 2207/30096 Tumor; Lesion
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/0014 using an image reference approach

Abstract

An embodiment of the present disclosure discloses a method and apparatus for annotating a medical image. An embodiment of the method comprises: acquiring a to-be-annotated medical image; annotating classification information for the to-be-annotated medical image, wherein the classification information comprises a category of a diagnosis result and a grade of the diagnosis result corresponding to the medical image; processing the to-be-annotated medical image using a pre-trained lesion area detection model, framing a lesion area in the to-be-annotated medical image, and annotating a lesion type of the lesion area, to enable the to-be-annotated medical image to be annotated with the lesion area and the lesion type of the lesion area; and splitting the framed lesion area from the to-be-annotated medical image with the framed lesion area to form a split image of the to-be-annotated medical image, to enable the to-be-annotated medical image to be annotated with the split image.

Claims

1. A method for annotating a medical image, comprising:

acquiring a to-be-annotated medical image;

annotating classification information for the to-be-annotated medical image, wherein the classification information comprises a category of a diagnosis result and a grade of the diagnosis result corresponding to the medical image;

processing the to-be-annotated medical image using a pre-trained lesion area detection model, framing a lesion area in the to-be-annotated medical image, and annotating a lesion type of the framed lesion area in the to-be-annotated medical image, to enable the to-be-annotated medical image to be annotated with the lesion area and the lesion type of the lesion area, wherein the lesion area detection model is used for framing the lesion area based on a position and size of the lesion area identified in the medical image, and annotating the lesion type of the lesion area; and

splitting the framed lesion area from the to-be-annotated medical image with the framed lesion area to form a split image of the framed lesion area in the to-be-annotated medical image, to enable the to-be-annotated medical image to be annotated with the split image.

2. The method according to claim 1, wherein the annotating classification information for the to-be-annotated medical image comprises:

processing the to-be-annotated medical image using a pre-trained image classification model to output the classification information of the to-be-annotated medical image, wherein the image classification model is used for annotating the classification information for the medical image.

3. The method according to claim 1, wherein the splitting the framed lesion area from the to-be-annotated medical image with the framed lesion area to form a split image of the to-be-annotated medical image comprises:

processing the to-be-annotated medical image with the framed lesion area using a pre-trained lesion area splitting model to output the split image of the framed lesion area in the to-be-annotated medical image, wherein the lesion area splitting model is used for obtaining the split image of the lesion area by splitting the medical image with the framed lesion area.

4. The method according to claim 2, further comprising training the image classification model;

wherein

the training the image classification model comprises;

acquiring a first medical image training set, the first medical image training set comprising a plurality of medical images and the classification information annotated on each of the medical images; and

obtaining the image classification model by training, using a convolutional neural network based on the first medical image training set.

5. The method according to claim 1, further comprising training the lesion area detection model;

wherein

the training the lesion area detection model comprises;

acquiring a second medical image training set, the second medical image training set comprising a plurality of medical images, lesion areas and lesion types of the lesion areas annotated on each of the medical images; and

obtaining the lesion area detection model by training, using a convolutional neural network based on the second medical image training set.

6. The method according to claim 3, further comprising training the lesion area splitting model;

wherein

the training the lesion area splitting model comprises;

acquiring a third medical image training set, the third medical image training set comprising a plurality of medical images with framed lesion areas and a split image of the lesion areas of the medical images with the framed lesion areas; and

obtaining the lesion area splitting model by training, using a convolutional neural network based on the third medical image training set.

7. The method according to claim 1, wherein after the processing the to-be-annotated medical image using a pre-trained lesion area detection model, framing a lesion area in the to-be-annotated medical image, and annotating the lesion type of the lesion area, the method further comprises:

outputting the to-be-annotated medical image, and the annotated lesion area and the lesion type of the lesion area in the to-be-annotated medical image to a client terminal, to enable a user to determine whether the annotated lesion area and the lesion type of the lesion area in the to-be-annotated medical image are correct; and

saving the annotated lesion area and the lesion type of the lesion area in the to-be-annotated medical image, if the annotated lesion area and the lesion type of the lesion area in the to-be-annotated medical image are correct;

or

receiving and saving a lesion area and a lesion type of the lesion area in the to-be-annotated medical image reannotated by the user, if the annotated lesion area and the lesion type of the lesion area in the to-be-annotated medical image are not correct.

8. The method according to claim 3, wherein before the outputting the split image of the framed lesion area in the to-be-annotated medical image, the method further comprises:

forming a pre-split area in the lesion area of the to-be-annotated medical image using the lesion area splitting model, and outputting the pre-split area to a client terminal, to enable a user to slightly adjust the pre-split area; and

receiving and saving the pre-split area slightly adjusted by the user.

9. The method according to claim 4, further comprising:

adding the to-be-annotated medical image and the classification information of the to-be-annotated medical image to the first medical image training set to retrain the image classification model.

10. The method according to claim 5, further comprising:

adding the to-be-annotated medical image, the annotated lesion area and the lesion type of the lesion area in the to-be-annotated medical image to the second medical image training set to retrain the lesion area detection model.

11. The method according to claim 6, further comprising:

adding the to-be-annotated medical image with the framed lesion area and the split image of the to-be-annotated medical image to the third medical image training set to retrain the lesion area splitting model.

12. An apparatus for annotating a medical image, comprising:

at least one processor; and

a memory storing instructions, the instructions when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising;

acquiring a to-be-annotated medical image;

annotating classification information for the to-be-annotated medical image, wherein the classification information comprises a category of a diagnosis result and a grade of the diagnosis result corresponding to the medical image;

processing the to-be-annotated medical image using a pre-trained lesion area detection model, framing a lesion area in the to-be-annotated medical image, and annotating a lesion type of the framed lesion area in the to-be-annotated medical image, to enable the to-be-annotated medical image to be annotated with the lesion area and the lesion type of the lesion area, wherein the lesion area detection model is used for framing the lesion area based on a position and size of the lesion area identified in the medical image, and annotating the lesion type of the lesion area; and

splitting the framed lesion area from the to-be-annotated medical image with the framed lesion area to form a split image of the framed lesion area in the to-be-annotated medical image, to enable the to-be-annotated medical image to be annotated with the split image.

13. The apparatus according to claim 12, wherein the annotating classification information for the to-be-annotated medical image comprises:

processing the to-be-annotated medical image using a pre-trained image classification model to output the classification information of the to-be-annotated medical image, wherein the image classification model is used for annotating a category of a diagnosis result and a grade of the diagnosis result for the medical image.

14. The apparatus according to claim 12, wherein the splitting the framed lesion area from the to-be-annotated medical image with the framed lesion area to form a split image of the to-be-annotated medical image comprises:

processing the to-be-annotated medical image with the framed lesion area using a pre-trained lesion area splitting model to output the split image of the framed lesion area in the to-be-annotated medical image, wherein the lesion area splitting model is used for obtaining the split image of the lesion area by splitting the medical image with the framed lesion area.

15. The apparatus according to claim 13, the operations further comprise training the image classification model;

wherein

the training the image classification model comprises;

acquiring a first medical image training set, the first medical image training set comprising a plurality of medical images and the classification information annotated on each of the medical images; and

obtaining the image classification model by training, using a convolutional neural network based on the first medical image training set.

16. The apparatus according to claim 12, the operations further comprise training the lesion area detection model;

wherein

the training the lesion area detection model comprises;

acquiring a second medical image training set, the second medical image training set comprising a plurality of medical images, lesion areas and lesion types of the lesion areas annotated on each of the medical images; and

obtaining the lesion area detection model by training, using a convolutional neural network based on the second medical image training set.

17. The apparatus according to claim 14, the operations further comprise training the lesion area splitting model;

wherein

the training the lesion area splitting model comprises;

acquiring a third medical image training set, the third medical image training set comprising a plurality of medical images with framed lesion areas and a split image of the lesion areas of the medical images with the framed lesion areas; and

obtaining the lesion area splitting model by training, using a convolutional neural network based on the third medical image training set.

18. The apparatus according to claim 12, wherein after the processing the to-be-annotated medical image using a pre-trained lesion area detection model, framing a lesion area in the to-be-annotated medical image, and annotating the lesion type of the lesion area, the operations further comprise:

outputting the to-be-annotated medical image, and the annotated lesion area and the lesion type of the lesion area in the to-be-annotated medical image to a client terminal, to enable a user to determine whether the annotated lesion area and the lesion type of the lesion area in the to-be-annotated medical image are correct; and

saving the annotated lesion area and the lesion type of the lesion area in the to-be-annotated medical image, if the annotated lesion area and the lesion type of the lesion area in the to-be-annotated medical image are correct;

or

receiving and saving a lesion area and a lesion type of the lesion area in the to-be-annotated medical image reannotated by the user, if the annotated lesion area and the lesion type of the lesion area in the to-be-annotated medical image are not correct.

19. The apparatus according to claim 14, wherein before the outputting the split image of the framed lesion area in the to-be-annotated medical image, the operations further comprise:

forming a pre-split area in the lesion area of the to-be-annotated medical image using the lesion area splitting model, and outputting the pre-split area to a client terminal, to enable a user to slightly adjust the pre-split area; and

receiving and saving the pre-split area slightly adjusted by the user.

20. The apparatus according to claim 15, the operations further comprise:

adding the to-be-annotated medical image and the classification information of the to-be-annotated medical image to the first medical image training set to retrain the image classification model.

21. The apparatus according to claim 16, the operations further comprise:

adding the to-be-annotated medical image, the annotated lesion area and the lesion type of the lesion area in the to-be-annotated medical image to the second medical image training set to retrain the lesion area detection model.

22. The apparatus according to claim 17, the operations further comprise:

adding the to-be-annotated medical image with the framed lesion area and the split image of the to-be-annotated medical image to the third medical image training set to retrain the lesion area splitting model.

23. A non-transitory computer storage medium storing a computer program, the computer program when executed by one or more processors, causes the one or more processors to perform operations, the operations comprising:

acquiring a to-be-annotated medical image;

annotating classification information for the to-be-annotated medical image, wherein the classification information comprises a category of a diagnosis result and a grade of the diagnosis result corresponding to the medical image;

processing the to-be-annotated medical image using a pre-trained lesion area detection model, framing a lesion area in the to-be-annotated medical image, and annotating a lesion type of the framed lesion area in the to-be-annotated medical image, to enable the to-be-annotated medical image to be annotated with the lesion area and the lesion type of the lesion area, wherein the lesion area detection model is used for framing the lesion area based on a position and size of the lesion area identified in the medical image, and annotating the lesion type of the lesion area; and

splitting the framed lesion area from the to-be-annotated medical image with the framed lesion area to form a split image of the framed lesion area in the to-be-annotated medical image, to enable the to-be-annotated medical image to be annotated with the split image.