System and method for visual annotation and knowledge representation

• US 7,453,472 B2
• Filed: 05/31/2003
• Issued: 11/18/2008
• Est. Priority Date: 05/31/2002
• Status: Active Grant

• Current Assignee: VisualShare, Inc. (XIFIN, Inc.), XIFIN, Inc.
• Original Assignee: University of Utah Research Foundation (State of Utah)
• Inventors: Cochella, Christopher, Lauman, Jason R., Goede, Patricia Anne
• Primary Examiner(s): Yang; Ryan R
• Application Number:
  US10/516,554
• Publication Number:
  US 20060061595A1
• Time in Patent Office: 1,998 Days
• Field of Search: 345/581, 345/634, 345/619, 345/629, 715/230, 715/713
• US Class Current: 345/634
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/56 having vectorial format
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G06T 2219/004 Annotating, labelling
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing

Abstract

A method and system for visually annotating an image. Annotations and notes to images, such as digital medical and healthcare images, may be stored in a structured vector representation alongside image information in a single, non-volatile and portable file or in a separate file from the image. The annotations may be composed of point, line and polygon drawings and text symbols, labels or definitions and captions or descriptions. The annotations may be structured in a manner that facilitates grouping and manipulation as user defined groups. The annotations may be related to an image but not inextricably bound such that the original image is completely preserved. Annotations may further be selectively displayed on the image for context appropriate viewing. The annotations may be retrieved for purposes such as editing, printing, display, indexing and reporting for example, and may be displayed on an image for interactive use with an embedded self-contained user interface.

Claims

1. A method of visually annotating a digital image, the method comprising:

presenting a digital image in a user interface of a display;

receiving information defining a region of interest identified by a user interacting with the presented digital image;

receiving textual information associated with the received information, wherein an annotation comprises the received information and the received textual information;

presenting the digital image overlaid with the received information and the received textual information;

presenting a plurality of annotation indicators in the user interface, an annotation indicator of the plurality of annotation indicators including the received textual information associated with the annotation;

receiving a selection of a first annotation indicator of the presented plurality of annotation indicators; and

presenting the digital image without the received information and the received textual information associated with the received selection.

2. The method of claim 1, wherein the digital image is an image of at least, a portion of a human body.

3. The method of claim 1, further comprising electronically storing the annotation as vector based information linked to the digital image.

4. The method of claim 3, wherein storing comprises storing the annotation in a second file and said digital image is stored in a first file.

5. The method of claim 3, wherein storing comprises storing the annotation in the same file which contains the digital image.

6. The method of claim 3 wherein storing comprises storing the annotation as text information.

7. The method of claim 6, wherein the annotation is stored in an extensible markup language compatible format.

8. The method of claim 3 wherein the annotation is stored in a format that can be electronically queried.

9. The method of claim 1, further comprising hierarchically organizing the annotations into logical groupings based on a user defined structure.

10. The method of claim 1, wherein at least part of the annotation utilizes user defined lexicons.

11. The method of claim 1, further comprising defining metadata associated with the annotation.

12. The method of claim 11 wherein the metadata is stored with the annotation.

13. The method of claim 11 wherein the metadata comprises at least one selected from the group consisting of:

the name of the author or creator of the annotation, a date indicating when the annotation was created, a title, a subject, a description, and an area of specialty of the author or creator.

14. The method of claim 1, wherein the region of interest is defined as an area selected from the group consisting of:

a point, a set of points, a polygon, and a polyline.

15. The method of claim 1 wherein the annotation can be displayed according to a predefined user grouping.

16. The method of claim 1 wherein the annotation is uniquely identified.

17. The method of claim 1 wherein the digital image comprises a sequence of digital images.

18. The method of claim 1, wherein the textual information includes a symbol and a label.

19. The method of claim 18, further comprising presenting the symbol or the label adjacent the defined region of interest in the user interface based on a user selection.

20. The method of claim 18, wherein the textual information further includes a caption.

21. The method of claim 1, further comprising receiving pointer information associated with the received information, wherein an annotation further comprises the received pointer information.

22. The method of claim 21, wherein the received pointer information includes secondary information related to the received information.

23. The method of claim 22, further comprising presenting the secondary information in the user interface based on activation of a hot-spot associated with a pointer created in the user interface based on the received pointer information.

24. The method of claim 1, further comprising:

receiving second information defining a second region of interest identified by the user interacting with the presented digital image;

receiving second textual information associated with the received second information, wherein a second annotation comprises the received second information and the received second textual information; and

associating the annotation and the second annotation with a first annotation group.

25. The method of claim 24, further comprising:

receiving a selection of the first annotation group to present overlaid on the displayed digital image; and

presenting the digital image with the annotation and the second annotation.

26. A device for visually annotating a digital image, the device comprising:

a processor; and

a computer memory operably coupled to the processor, the computer memory comprising programming code that, upon execution by the processor, performs operations comprising

presenting a digital image in a user interface of a display;

receiving information defining a region of interest identified by a user interacting with the presented digital image;

receiving textual information associated with the received information, wherein an annotation comprises the received information and the received textual information;

presenting the digital image overlaid with the received information and the received textual information;

presenting a plurality of annotation indicators in the user interface, an annotation indicator of the plurality of annotation indicators including the received textual information associated with the annotation;

receiving a selection of a first annotation indicator of the presented plurality of annotation indicators; and

presenting the digital image without the received information and the received textual information associated with the received selection.

27. The device of claim 26, wherein the programming code further performs operations comprising:

receiving second information defining a second region of interest identified by the user interacting with the presented digital image;

receiving second textual information associated with the received second information, wherein a second annotation comprises the received second information and the received second textual information; and

associating the annotation and the second annotation with a first annotation group.

28. The device of claim 27, wherein the programming code further performs operations comprising:

receiving a selection of the first annotation group to present overlaid on the displayed digital image; and

presenting the digital image with the annotation and the second annotation.

29. A computer memory comprising programming code therein that, upon execution by a processor, causes a computer to:

present a digital image in a user interface of a display;

receive information defining a region of interest identified by a user interacting with the presented digital image;

receive textual information associated with the received information, wherein an annotation comprises the received information and the received textual information;

present the digital image overlaid with the received information and the received textual information;

present a plurality of annotation indicators in the user interface, an annotation indicator of the plurality of annotation indicators including the received textual information associated with the annotation;

receive a selection of a first annotation indicator of the presented plurality of annotation indicators; and

present the digital image without the received information and the received textual information associated with the received selection.

30. The computer memory of claim 29, wherein the programming code further causes the computer to define metadata associated with the annotation.

31. The computer memory of claim 29, wherein the programming code further causes the computer to store the annotation as vector based information linked to the digital image.

32. The computer memory of claim 29, wherein the textual information includes a symbol and a label.

33. The computer memory of claim 32, wherein the programming code further causes the computer to present the symbol or the label adjacent the defined region of interest in the user interface based on a user selection.

34. The computer memory of claim 32, wherein the textual information further includes a caption.

35. The computer memory of claim 29, wherein the programming code further causes the computer to receive pointer information associated with the received information, wherein an annotation further comprises the received pointer information.

36. The computer memory of claim 35, wherein the received pointer information includes secondary information related to the received information.

37. The computer memory of claim 36, wherein the programming code further causes the computer to present the secondary information in the user interface based on activation of a hot-spot associated with a pointer created in the user interface based on the received pointer information.

38. The computer memory of claim 29, wherein the programming code further causes the computer to:

receive second information defining a second region of interest identified by the user interacting with the presented digital image;

receive second textual information associated with the received second information, wherein a second annotation comprises the received second information and the received second textual information; and

associate the annotation and the second annotation with a first annotation group.

39. The computer memory of claim 38, wherein the programming code further causes the computer to:

receive a selection of the first annotation group to present overlaid on the displayed digital image; and

present the digital image with the annotation and the second annotation.