Systems and methods for controlling user repeatability and reproducibility of automated image annotation correction

• US 9,589,349 B2
• Filed: 08/27/2014
• Issued: 03/07/2017
• Est. Priority Date: 09/25/2013
• Status: Active Grant

• Current Assignee: Heartflow Inc.
• Original Assignee: Heartflow Inc.
• Inventors: Grady, Leo, Schaap, Michiel
• Primary Examiner(s): Allison, Andrae S
• Application Number:
  US14/470,314
• Publication Number:
  US 20150086133A1
• Time in Patent Office: 923 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing

Abstract

Systems and methods are disclosed for controlling image annotation. One method includes acquiring a digital representation of image data and generating a set of image annotations for the digital representation of the image data. The method also may include determining an association between members of the set of image annotations and generating one or more groups of members based on the association. A representative annotation from the one or more groups may also be determined, presented for selection, and the selection may be recorded in memory.

Claims

1. A computer-implemented method of controlling image annotation, using at least one computer system, the method comprising using a processor for:

receiving, at the at least one computer system, a digital representation of image data;

generating, by the at least one computer system, a plurality of image annotations of anatomic features identified within the image data;

determining, by the at least one computer system, one or more associations between members of a set of image annotations of the plurality of image annotations;

generating one or more groups of image annotations, each group comprising one or more image annotations among the generated plurality of image annotations and each group being generated based on an association between the image annotations of the group;

determining, by the at least one computer system, a representative annotation for each group selected from among the image annotations of the group, the representative annotation being determined as an annotation among the image annotations of the group closest to an aggregation of the image annotations of the group and the representative annotation identifying anatomic features identified within the image data; and

presenting, by the at least one computer system, the representative annotation of each group of the one or more groups for selection or correction by a user.

2. The computer-implemented method of claim 1, further comprising the following steps prior to the step of generating a plurality of image annotations of features identified within the image data:

presenting an automated annotation of the digital representation for validation and/or correction; and

receiving a user input of an image annotation based on the user'"'"'s evaluation of the automated annotation.

3. The computer-implemented method of claim 2, wherein the image data is a portion of a body organ.

4. The computer-implemented method of claim 1, wherein the digital representation of image data is acquired electronically via a network.

5. The computer-implemented method of claim 1, wherein the set of image annotations are electronically automatically generated.

6. The computer-implemented method of claim 5, wherein the set of image annotations are generated using multiple image analysis algorithms.

7. The computer-implemented method of claim 1, wherein each of the one or more associations between members of the set of image annotations is determined by assigning a similarity score between the members.

8. The computer-implemented method of claim 1, wherein each of the one or more associations between members of the set of image annotations is determined by assigning a pair of image annotations among the set of image annotations to a group if the pair of image annotations are the same.

9. The computer-implemented method of claim 1, wherein the automated image annotation comprises a box around a target feature.

10. The computer-implemented method of claim 9, further comprising applying a k-means algorithm to a centroid of the box.

11. A system of controlling image annotation, the system comprising:

a data storage device storing instructions for controlling image annotation; and

a processor configured to execute the instruction to perform a method including;

receiving a digital representation of image data;

generating a plurality of image annotations of anatomic features identified within the image data;

determining one or more associations between members of a set of image annotations of the plurality of image annotations;

generating one or more groups of image annotations, each group comprising one or more image annotations among the generated plurality of image annotations and each group being generated based on an association between the image annotations of the group;

determining a representative annotation for each group selected from among the image annotations of the group, the representative annotation being determined as an annotation among the image annotations of the group closest to an aggregation of the image annotations of the group and the representative annotation identifying anatomic features identified within the image data; and

presenting the representative annotation of each group of the one or more groups for selection or correction by a user.

12. The system of claim 11, further comprising the following steps prior to the step of generating a set of image annotations of features identified within the image data:

presenting an automated annotation of the digital representation for validation and/or correction; and

receiving a user input of an image annotation based on the user'"'"'s evaluation of the automated annotation.

13. The system of claim 11, wherein the processor is further configured to electronically acquire the digital representation of image data via a network.

14. The system of claim 11, wherein the processor is further configured to automatically generate the set of image annotations.

15. The system of claim 14, wherein the processor is further configured to automatically generate the set of image annotations using multiple image analysis algorithms.

16. The system of claim 11, wherein the processor in configured to determine each of the one or more associations between members of the set of image annotations by assigning a similarity score between the members.

17. The system of claim 11, wherein processor is configured to determine each of the one or more associations between members of the set of image annotations by assigning a pair of image annotations among the set of image annotations to a group if the pair of image annotations are the same.

18. The system of claim 11, wherein the processor is configured to generate a box around a target feature.

19. A non-transitory computer-readable medium for use on a computer system containing computer-executable programming instructions for controlling image annotation, the instructions being executable by the computer system for:

receiving a digital representation of image data;

generating a plurality of image annotations of anatomic features identified within the image data;

determining one or more associations between members of a set of image annotations of the plurality of image annotations;

generating one or more groups of image annotations, each group comprising one or more image annotations among the generated plurality of image annotations and each group being generated based on an association between the image annotations of the group;

determining a representative annotation for each group selected from among the image annotations of the group, the representative annotation being determined as an annotation among the image annotations of the group closest to an aggregation of the image annotations of the group and the representative annotation identifying anatomic features identified within the image data; and

presenting the representative annotation of each group of the one or more groups for selection or correction by a user.

20. The non-transitory computer readable medium of claim 19, wherein each of the one or more associations between members of the set of image annotations is determined by assigning a similarity score between the members.