Method and apparatus for archiving and visualizing digital images

• US 7,970,240 B1
• Filed: 12/17/2002
• Issued: 06/28/2011
• Est. Priority Date: 12/17/2001
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Sturtevant, Reed P. M., Herf, Michael B., Chao, Edward K., Perkins, Lars D.
• Primary Examiner(s): TUCKER, WESLEY J
• Application Number:
  US10/321,403
• Time in Patent Office: 3,115 Days
• Field of Search: 382305-307
• US Class Current: 382/305
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 7/08 Sorting, i.e. grouping record carriers in numerical or other ordered sequence according to the classification of at least some of the information they carry by merging two or more sets of carriers in ordered sequence G06F7/16

Abstract

A computer based, digital image management system that visually presents an entire archive in a manner that facilitates easy, non-textual, searching for specific images. The system consists of two main elements; a file management methodology, and a graphical user interface. The file management method utilizes reference files that contain multiple properties (metadata) relating to each image file. Image files are grouped into a multiplicity of collections based on user-defined associations of the properties. The graphical user interface visually represents the groups of image files on 2-dimensional or 3-dimensional graphic metaphors. In addition, the graphical interface uses multiple display variables such as icons, mnemonics, size, movement, color, and luminance to signify relationships between groups, and/or communicate key properties of individual or groups of images.

Claims

1. A method of managing digital images using an electronic controller comprising:

acquiring and storing image data;

acquiring and storing metadata related to images represented by the image data;

grouping the images into one or more collections using only the metadata; and

causing the grouped images to be accessed using visual cues graphically illustrating one or more characteristics within the relevant metadata, the characteristics including characteristics of user interaction with the grouped images, the characteristics of user interaction with the grouped images including information related to a click stream of the user, wherein the click stream of the user is generated using at least one of a frequency of access of the grouped images by the user or a characteristic of access of the grouped images by the user.

2. The method of claim 1 wherein acquiring and storing metadata comprises storing characteristics of the environment where the image was captured.

3. The method of claim 1 wherein acquiring and storing metadata comprises storing characteristics of an image'"'"'s content.

4. The method of claim 1 wherein acquiring and storing metadata comprises storing characteristics of a user'"'"'s interaction with an image.

5. The method of claim 1 wherein acquiring and storing metadata comprises:

storing temporal metadata that indicates the time at which the image data was acquired and associating the metadata with the image data.

6. The method of claim 1 wherein acquiring and storing metadata comprises:

storing location metadata that indicates the location of the digital image'"'"'s acquisition and associating the metadata with the image data.

7. The method of claim 1 wherein acquiring and storing metadata comprises:

storing metadata that indicates the lighting conditions in the scene of which the image is acquired and associating the metadata with the image data.

8. The method of claim 1 wherein acquiring and storing metadata comprises:

storing metadata related to a geometric pattern within the image and associating the metadata with the image data.

9. The method of claim 1 wherein acquiring and storing metadata comprises:

storing metadata related to human identification and associating the metadata with the image data.

10. The method of claim 1 wherein acquiring and storing metadata comprises:

storing temporal metadata related to the time the image data is first stored and associating the temporal metadata to the stored image data.

11. The method of claim 1 wherein acquiring and storing metadata comprises:

storing metadata indicating the frequency with which a file containing the image data is edited and associating that metadata with the stored image data.

12. The method of claim 1 wherein acquiring and storing metadata comprises:

storing metadata indicating the frequency with which a file containing the image data is accessed and associating that metadata with the stored image data.

13. The method of claim 1 wherein acquiring and storing metadata comprises:

storing metadata indicating the frequency with which the image data is exported from a system in which the image data is stored and associating that metadata with the stored image data.

14. The method of claim 1 wherein grouping the images comprises:

comparing an element of an image'"'"'s metadata against a constraint.

15. The method of claim 14 wherein comparing an element of an image'"'"'s metadata comprises:

comparing the time of an image'"'"'s capture to a time span and grouping images having times of capture within the time span.

16. The method of claim 14 wherein comparing an element of an image'"'"'s metadata comprises:

comparing the aspect ratio of the image to a known value to determine whether the image is a photograph.

17. The method of claim 14 wherein comparing an element of an image'"'"'s metadata comprises:

comparing the resolution of the image to a known value to determine whether the image is a photograph.

18. The method of claim 14 wherein comparing an element of an image'"'"'s metadata comprises:

comparing the distribution of colors within an image to a known distribution of colors and grouping images having color distributions within a predetermined range of one another.

19. The method of claim 18 wherein comparing an element of an image'"'"'s metadata comprises:

comparing the color temperature of an image to a range of color temperatures and grouping images having color temperatures within the same span.

20. The method of claim 1 further comprising:

causing iconic representations of the images arranged in clusters to be displayed in a first display segment; and

responding to a user'"'"'s selection of one of the iconic representations displayed in the first display segment by copying the iconic representation of the image into a second display segment to facilitate a selection of the associated image files for photo processing.

21. The method of claim 1 further comprising:

causing iconic representations of the images arranged in clusters to be displayed in a first display segment;

responding to a user'"'"'s selection of one of the iconic representations displayed in the first display segment by copying the iconic representation of the image into a second display segment; and

passing image files associated with the copied iconic representations to an integrated email application for emailing.

22. The method of claim 21 wherein causing iconic representations of the images to be displayed comprises:

displaying a thumbnail image of an associated image.

23. The method of claim 21 wherein causing iconic representations of the images to be displayed comprises:

displaying a file icon.

24. The method of claim 1 wherein the visual cues are represented using a timeline theme.

25. The method of claim 1 wherein visual cues are represented using a map theme.

26. The method of claim 1 wherein the visual cues are represented using a human relationship theme.

27. The method of claim 1, wherein the information related to a click stream of the user includes at least one of:

a time the grouped images were acquired, a frequency of editing the grouped images, a characteristic of editing the grouped images, a frequency of retrieval of the grouped images from an archive, a characteristic of retrieval of the grouped images from the archive, a frequency of the grouped images being sent through a network, a characteristic of the grouped images being sent through the network, a frequency of the grouped images being printed, a characteristic of the grouped images being printed, a frequency of the grouped images being stored onto permanent media, and a characteristic of the grouped images being stored onto permanent media.

28. An apparatus comprising:

a storage device to store image data and metadata related to images represented by the image data; and

an electronic controller to associate the metadata with the image data, to group the images into one or more collections using only the metadata, and to cause the grouped images to be accessed using visual cues graphically illustrating one or more characteristics within the relevant metadata, the characteristics including characteristics of user interaction with the grouped images, the characteristics of user interaction with the grouped images including information related to a click stream of the user, wherein the click stream of the user is generated using at least one of a frequency of access of the grouped images by the user or a characteristic of access of the grouped images by the user.

29. The apparatus of claim 28 wherein the metadata comprises location metadata that indicates location of image acquisition and/or temporal metadata that indicates the time at which the image data was acquired.

30. The apparatus of claim 28 wherein the metadata comprises metadata that indicates lighting conditions in the scene of which the image is acquired.

31. The apparatus of claim 28 wherein the electronic controller is to group an image by comparing an element of an image'"'"'s metadata against a constraint.

32. The apparatus of claim 31 wherein the electronic controller is further to compare the time of an image'"'"'s capture to a time span and to group images having times of capture within the time span.

33. The apparatus of claim 31 wherein the electronic controller is to group images by comparing the color temperature of an image to a range of color temperatures and grouping images having color temperatures within the same span.

34. The apparatus of claim 31 wherein the electronic controller is to group images by matching a metadata element associated with at least one image file to a reference point.

35. The apparatus of claim 28 wherein the electronic controller is:

to cause iconic representations of the images arranged in clusters to be displayed in a first display segment, and

to respond to a user'"'"'s selection of one of the iconic representations displayed in the first display segment by copying the iconic representation of the image into a second display segment to facilitate a selection of the associated image files for photo-processing ordering.

36. The apparatus of claim 28 wherein the electronic controller is further to cause iconic representations of the images arranged in clusters to be displayed in a first display segment,

to respond to a user'"'"'s selection of one of the iconic representations displayed in the first display segment by copying the iconic representation of the image into a second display segment, and

to pass image files associated with the copied iconic representations to an integrated email application for emailing.

37. The apparatus of claim 36 wherein the electronic controller is further to display a thumbnail image of an associated image and to display a file icon.

38. The apparatus of claim 28, wherein the information related to a click stream of the user includes at least one of:

a time the grouped images were acquired, a frequency of editing the grouped images, a characteristic of editing the grouped images, a frequency of retrieval of the grouped images from an archive, a characteristic of retrieval of the grouped images from the archive, a frequency of the grouped images being sent through a network, a characteristic of the grouped images being sent through the network, a frequency of the grouped images being printed, a characteristic of the grouped images being printed, a frequency of the grouped images being stored onto permanent media, and a characteristic of the grouped images being stored onto permanent media.

39. A computer readable storage medium having instructions, which when executed on a processing system, cause the processing system to perform a method comprising:

acquiring and storing image data;

acquiring and storing metadata related to images represented by the image data;

grouping the images into one or more collections using only the metadata; and

causing the grouped images to be accessed using visual cues graphically illustrating one or more characteristics within the relevant metadata, the characteristics including characteristics of user interaction with the grouped images, the characteristics of user interaction with the grouped images including information related to a click stream of the user, wherein the click stream of the user is generated using at least one of a frequency of access of the grouped images by the user or a characteristic of access of the grouped images by the user.

40. The medium of claim 39 wherein the metadata comprises location metadata that indicates the location of image acquisition and/or temporal metadata that indicates the time at which the image data was acquired.

41. The medium of claim 39 wherein the metadata comprises metadata that indicates the lighting conditions in the scene of which the image is acquired.

42. The computer readable storage medium of claim 39 wherein the method further comprises:

causing iconic representations of the images arranged in clusters to be displayed in a first display segment; and

responding to a user'"'"'s selection of one of the iconic representations displayed in the first display segment by copying the iconic representation of the image into a second display segment to facilitate a selection of the associated image files for photo-processing ordering.

43. The computer readable medium of claim 39 wherein the method further comprises:

causing iconic representations of the images arranged in clusters to be displayed in a first display segment;

responding to a user'"'"'s selection of one of the iconic representations displayed in the first display segment by copying the iconic representation of the image into a second display segment; and

passing image files associated with the copied iconic representations to an integrated email application for emailing.

44. The computer-readable storage medium of claim 39, wherein the information related to a click stream of the user includes at least one of:

a time the grouped images were acquired, a frequency of editing the grouped images, a characteristic of editing the grouped images, a frequency of retrieval of the grouped images from an archive, a characteristic of retrieval of the grouped images from the archive, a frequency of the grouped images being sent through a network, a characteristic of the grouped images being sent through the network, a frequency of the grouped images being printed, a characteristic of the grouped images being printed, a frequency of the grouped images being stored onto permanent media, and a characteristic of the grouped images being stored onto permanent media.