Methods for the electronic annotation, retrieval, and use of electronic images

• US 7,010,751 B2
• Filed: 10/15/2001
• Issued: 03/07/2006
• Est. Priority Date: 02/18/2000
• Status: Expired due to Term

• Current Assignee: Mind Fusion, LLC (Ascend Innovation Management, LLC)
• Original Assignee: University of Maryland, College Park
• Inventors: Shneiderman, Ben A.
• Primary Examiner(s): Luu, Sy D.
• Application Number:
  US09/976,362
• Publication Number:
  US 20020054059A1
• Time in Patent Office: 1,604 Days
• Field of Search: 345/700, 345/708, 345/751, 345/764, 345/769, 345/968, 715/104, 715/500, 715/512, 715/514, 715/700, 715/708, 715/716, 715723-726, 715751-759, 715/764, 715/769, 715/968, 715/731, 715/810
• US Class Current: 715/232
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 715/968 interface for database querying and retrieval

Abstract

The present invention provides software for electronically annotating electronic images, such as drawings, photographs, video, etc., through the drag-and-drop of annotations from a pre-defined, but extendable, list. The annotations are placed at a user-selected X, Y location on the image, and stored in a searchable database. Thus, they can be searched in order to retrieve, organize, group, or display desired electronic images or collections of such images. The annotations may be text, images, sounds, etc. The invention provides a flexible, easy to learn, rapid, low error rate and satisfying interface for accomplishing such tasks.

Claims

1. A computer software-mediated method for annotating a user-selected target image aspect contained within an electronic image that comprises a plurality of inherently unrelated image aspects, in which the placement of all desired image annotations is not determined solely by the scale of the image, and the X, Y coordinates of the image'"'"'s center, wherein said inherently unrelated image aspects include a target image aspect and a non-target image aspect, said method comprising:

(A) displaying said electronic image using a computer-generated display interface;

(B) accessing and querying a searchable electronic list of annotation terms for a desired annotation term, and permitting a user to add or edit said desired annotation term to said database list;

(C) associating said desired accessed or added or edited annotation term of said electronic list with said selected target image aspect of said electronic image, wherein said associating comprises positioning said desired annotation term at a user-selected location within the displayed electronic image to identify said selected target image aspect and distinguish it from a non-target image aspect contained within said electronic image, wherein said positioning is accomplished by selecting dragging and dropping said desired annotation term to said user-selected location;

(D) storing said accessed or added or edited annotation term and said electronic image containing said target image aspect in a searchable electronic database of annotation terms and electronic images; and

(E) permitting a subsequent user to display an electronic image (D) containing said selected target image aspect upon the querying of said electronic database (D) for said associated annotation term, wherein said displayed electronic image (D) optionally displays said annotation term positioned to identify the associated target image aspect and distinguish it from non-target image aspects contained within said displayed electronic image (D), and permitting said subsequent user to add, or edit; and

store a desired annotation term to said database list and to associate said new desired annotation term with an image aspect of said displayed electronic image.

2. The computer software-mediated method of claim 1, wherein said computer is a personal computer having a screen, and wherein said displaying comprises displaying the displayed image on the screen of the personal computer.

3. The computer software-mediated method of claim 1, wherein said annotation is a textual annotation.

4. The computer software-mediated method of claim 1, further comprising selecting said image from the group consisting of a photograph, a writing, and a drawing.

5. The computer software-mediated method of claim 4, wherein said electronic image is a photograph.

6. The computer software-mediated method of claim 5, wherein said image aspect to be annotated is a person depicted in said photograph.

7. The computer software-mediated method of claim 6, wherein said annotation is a name of said person depicted in said photograph.

8. The computer software-mediated method of claim 1, that further permits the user to selectively hide and re-display the annotation term positioned within the electronic image.

9. A computer software-mediated method for retrieving and displaying an electronic image containing a target image aspect;

wherein;

said electronic image comprises a plurality of inherently unrelated image aspects, in which the placement of all desired image annotations is not determined solely by the scale of the image, and the X, Y coordinates of the image'"'"'s center, wherein said inherently unrelated image aspects include a target image aspect and a non-target image aspect, and is stored in a database of electronic images and annotation terms,

said target image aspect is associated with an annotation term of said database, said associated annotation terms being displayable positioned at a location within the displayed electronic image that identifies said selected target image aspect and distinguishes it from non-target image aspects contained within said electronic image, said method comprising;

(A) accessing and querying said database of electronic images and annotation terms for electronic images containing a target image aspect associated with a queried annotation term;

(B) retrieving from said database an electronic image identified through such querying that contains said target image aspect;

(C) displaying said retrieved electronic image, wherein said retrieved image displays said queried annotation term positioned to identify the associated target image aspect and distinguish it from non-target image aspects contained within the displayed electronic image;

(D) permitting a user to add or edit said desired annotation term to said database list;

(E) storing said accessed or added or edited annotation term and said electronic image containing said target image aspect in a searchable electronic database of annotation terms and electronic images; and

(F) permitting a subsequent user to display said electronic image (E) containing said selected target image aspect upon the querying of said electronic database (E) for said associated annotation term, wherein said displayed electronic image (E) optionally displays said annotation term positioned to identify the associated target image aspect and distinguish it from non-target image aspects contained within said displayed electronic image (E), and permitting said subsequent user to add, or edit, and store a new desired annotation term to said database list, and to associate said new desired annotation term with an image aspect of said displayed electronic image by selecting dragging and dropping said new desired annotation term to a user-selected location.

10. The computer software-mediated method of claim 9, wherein in said step (C), said image is displayed on the screen of a personal computer.

11. The computer software-mediated method of claim 9, wherein said annotation is a textual annotation.

12. The computer software-mediated method of claim 9, wherein said electronic image is selected from the group consisting of a photograph, a writing, and a drawing.

13. The computer software-mediated method of claim 12, wherein said electronic image is a photograph.

14. The computer software-mediated method of claim 13, wherein said desired image aspect is a person depicted in said photograph.

15. The computer software-mediated method of claim 14, wherein said annotation is a name of said person depicted in said photograph.

16. The computer software-mediated method of claim 9, that further permits the user to selectively hide and re-display the annotation term positioned within the electronic image.

17. A computer software-mediated method for retrieving and displaying electronic images containing related target image aspects, wherein:

said electronic images each comprise a plurality of inherently unrelated image aspects, in which the placement of all desired image annotations is not determined solely by the scale of the image, and the X, Y coordinates of the image'"'"'s center, wherein said inherently unrelated image aspects include a target image aspect and a non-target image aspect, and are stored in a database of electronic images and annotation terms,

said related target image aspects are each associated with at least one member of a family of related annotation terms of said database, each of said associated annotation terms being displayable positioned at a location within the displayed electronic image that identifies said selected target image aspect and distinguishes it from non-target image aspects contained within said electronic image, said method comprising;

(A) accessing and querying said database of electronic images and annotation terms for electronic images containing a target image aspect associated with any of said family of related annotation terms;

(B) retrieving from said database electronic images identified through such querying that contain a target image aspect;

(C) displaying a retrieved electronic image, wherein said retrieved image displays a queued annotation term positioned to identify the associated target image aspect and distinguish it from non-target image aspects contained within the displayed electronic image;

(D) permitting a user to add or edit said desired annotation term to said database list by selecting, dragging and dropping said desired annotation term to a user-selected location;

(E) storing said accessed or added or edited annotation terms and said electronic images containing said target image aspects in a searchable electronic database of annotation terms and electronic images; and

(F) permitting a subsequent user to display said electronic images (D) containing said selected target image aspects upon the querying of said electronic database (D) for electronic images containing a target image aspect associated with any of said family of related annotation terms, wherein a displayed electronic image (D) optionally displays said annotation term positioned to identify the associated target image aspect and distinguish it from non-target image aspects contained within said displayed electronic image (D), and permitting said subsequent user to add, or edit; and

store a new desired annotation term to said database list; and

to associate said new desired annotation term with an image aspect of said displayed electronic image.

18. The computer software-mediated method of claim 17, wherein in said step (C), said image is displayed on the screen of a personal computer.

19. The computer software-mediated method of claim 17, wherein said annotation is a textual annotation.

20. The computer software-mediated method of claim 17, wherein said electronic image is selected from the group consisting of a photograph, a writing, and a drawing.

21. The computer software-mediated method of claim 20, wherein said electronic image is a photograph.

22. The computer software-mediated method of claim 21, wherein said desired image aspect is a person depicted in said photograph.

23. The computer software-mediated method of claim 22, wherein said annotation is a name of said person depicted in said photograph.

24. The computer software-mediated method of claim 17, that further permits the user to selectively hide and re-display the annotation term positioned within an electronic image.

25. A computer system specially adapted by software to store, retrieve and display an electronic image containing a target image aspect;

wherein;

said electronic image comprises a plurality of inherently unrelated image aspects, in which the placement of all desired image annotations is not determined solely by the scale of the image, and the X, Y coordinates of the image'"'"'s center, wherein said inherently unrelated image aspects include a target image aspect and a non-target image aspect, and is stored in a database of electronic images and annotation terms,

said target image aspect is associated with an annotation term of said database, said associated annotation terms being displayable positioned at a location within the displayed electronic image that identifies said selected target image aspect and distinguishes it from non-target image aspects contained within said electronic image, said computer system accomplishing the;

(A) accession and querying of said database of electronic images and annotation terms for electronic images containing a target image aspect associated with a queried annotation term upon the request of

(B) retrieval from said database of an electronic image identified through such querying that contains said target image aspect;

(C) display of said retrieved electronic image, wherein said retrieved image displays said queried annotation term positioned to identify the associated target image aspect and distinguish it from non-target image aspects contained within the displayed electronic image;

(D) addition or editing of a desired annotation term to said database list by a user, and associating of a desired annotation term with an image aspect by selecting dragging and dropping said desired annotation term to a user-selected location in said electronic image;

(E) storage of said accessed or added or edited annotation term and said electronic image containing said target image aspect in a searchable electronic database of annotation terms and electronic images;

(F) display of said electronic image (E) containing said selected target image aspect to a subsequent user upon the querying of said electronic database (E) for said associated annotation term either alone or in combination with a different annotation term, wherein said displayed electronic image (E) optionally displays said annotation term positioned to identify the associated target image aspect and distinguish it from non-target image aspects contained within said displayed electronic image (E); and

(G) addition or editing and storage of a new desired annotation term to said database list by a subsequent user, and the association of said new desired annotation term with an image aspect of said displayed electronic image.

26. The computer system of claim 25, wherein said annotation is a textual annotation.