Agent for integrated annotation and retrieval of images

• US 7,028,253 B1
• Filed: 10/10/2000
• Issued: 04/11/2006
• Est. Priority Date: 10/10/2000
• Status: Expired due to Fees

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Wood, Mark D., Rosenzweig, Elizabeth, Lieberman, Henry, Singh, Pushpinder
• Primary Examiner(s): Hong, Stephen
• Assistant Examiner(s): LUDWIG, MATTHEW J
• Application Number:
  US09/685,112
• Time in Patent Office: 2,009 Days
• Field of Search: 715/531, 715/526, 715/512, 715/529, 715/731, 715/752, 707/4, 345/625, 345/634
• US Class Current: 715/232
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation

Abstract

A method for integrated retrieval and annotation of stored images involves running a user application in which text entered by a user is continuously monitored to isolate the context expressed by the text. The context is matched with metadata associated with the stored images, thereby providing one or more matched images, and the matched images are retrieved and displayed in proximity with the text. The context is then utilized to provide suggested annotations to the user for the matched images, together with the capability of selecting certain of the suggested annotations for subsequent association with the matched images. In a further extension, the method provides the user with the capability of inserting selected ones of the matched images into the text of the application, and further provides for automatically updating the metadata for the matched images.

Claims

1. A method for utilizing images that are stored in a data base with an e-mail application in which text is input by a user into a message, said method comprising the steps of:

a) providing metadata associated with the stored images;

b) running an e-mail application in which text is entered by a user into a message window;

c) continuously monitoring text typed by the user into the message window to isolate keywords in the text;

d) matching and ranking the keywords with the metadata associated with the stored images, thereby providing one or more matched images that are ranked according to the keywords;

e) retrieving and displaying the matched images alongside the message window in order of their rank; and

f) providing the user with the capability of moving selected ones of the matched images into the message window and utilizing the context to automatically provide new keyword annotations for the matched images.

2. The method as claimed in claim 1, wherein the step of utilizing the context further includes automatically updating the metadata for the matched image with the annotations.

3. The method as claimed in claim 1, wherein the ranking is based on the number of matched keywords for each matched image.

4. The method as claimed in claim 1, wherein the keywords and metadata include temporal references and the ranking is based on the temporal distance between the keyword and the metadata associated with each matched image.

5. The method as claimed in claim 1, further comprising the step of utilizing the context to either automatically or manually select new keyword annotations for the matched images, wherein the ranking is based on a higher confidence level for the manually selected keyword annotations.

6. The method as claimed in claim 1, wherein the keywords and metadata include spatial or geographic references and the ranking is based on the spatial or geographic correspondence between the keyword and the metadata associated with each matched image.

7. A method for utilizing images that are stored in a data base with an e-mail application in which text is input by a user into a message, said method comprising the steps of:

a) providing metadata associated with the stored images;

b) running an e-mail application in which text is entered by a user into a message window;

c) continuously monitoring text typed by the user into the message window to isolate keywords in the text;

d) matching and ranking the keywords with the metadata associated with the stored images thereby providing one or more matched images that are ranked according to the keywords;

e) retrieving and displaying the matched images alongside the message window in order of their rank; and

f) providing the user with the capability of moving selected ones of the matched images into the message window and utilizing the context to provide new keyword annotations for the matched images, together with the capability of manually selecting certain of the annotations for subsequent association with the matched images in the data base.

8. The method as claimed in claim 7, wherein the step of utilizing the context further includes automatically updating the metadata for the matched image with the annotations.

9. A computer implemented autonomous agent stored on a computer readable medium and including programming instructions for use in interfacing a text-based application with a picture archive containing representations of images and metadata associated with the images said agent comprising:

a language analyzer for continuously monitoring text entered by the user into the application and extracting keywords appropriate to the context surrounding the entered text;

a retrieval agent for retrieving images from the archive on the basis of one or more matches between the extracted keywords and the metadata;

an annotation agent for providing suggested annotations to the user for the matched images based on the extracted keywords, together with the capability of selecting certain of the suggested annotations for subsequent association with the matched images; and

a picture database viewer for displaying the retrieved images to the user and for enabling the images to be incorporated into the application, as desired by the user.

10. The computer implemented autonomous agent stored on a computer readable medium as claimed in claim 9, wherein the retrieval agent ranks the retrieved images based on attributes of the keywords, and the picture database viewer displays the retrieved images in order of their ranking.

11. The computer implemented autonomous agent stored on a computer readable medium as claimed in claim 9, wherein the application is an e-mail application.

12. The computer implemented autonomous agent stored on a computer readable medium as claimed in claim 9, wherein the picture database viewer incorporates a retrieved image into the application by a drag and drop operation.

13. The computer implemented autonomous agent stored on a computer readable medium as claimed in claim 9, wherein the picture database viewer incorporates a retrieved image into the application by a one click selection of the image.

14. The computer implemented autonomous agent stored on a computer readable medium as claimed in claim 9, further comprising a manual annotation agent for adding annotations to the retrieved image and for storing the added annotations to the metadata stored in the archive.

15. The computer implemented autonomous agent stored on a computer readable medium as claimed in claim 9, wherein the retrieval agent retrieves images from the archive on the basis of matching between the extracted keywords and temporal metadata identifying the time and date when the images were captured.

16. The computer implemented autonomous agent stored on a computer readable medium as claimed in claim 9, further comprising an automatic import agent for automatically entering images into the archive from one or more specified image sources.

17. The computer implemented autonomous agent stored on a computer readable medium as claimed in claim 16, wherein the retrieval agent automatically retrieves the entered images and the picture database viewer automatically displays the entered images.