Automatically organizing images

• US 10,078,781 B2
• Filed: 06/13/2014
• Issued: 09/18/2018
• Est. Priority Date: 06/13/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Meyer, Cayden
• Primary Examiner(s): Lefkowitz, Sumati
• Assistant Examiner(s): Sun, Jiangeng
• Application Number:
  US14/303,997
• Publication Number:
  US 20150363640A1
• Time in Patent Office: 1,558 Days
• Field of Search: None
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables

Abstract

A method includes obtaining a plurality of images, including geolocation information and time information for each image. The method also includes identifying a group of related images from the plurality of images from a common geographical area and a common time period based on the geolocation information and the time information for each of the plurality of images, and identifying an event that occurred within the common time period and within the common geographical area by searching a repository of event information. The method also includes storing the group of related images in an image collection, and assigning a title to the image collection, wherein the title is based at least in part on a title of the event.

Claims

1. A method for organizing digital images by events, the method comprising:

extracting, by a computing device, metadata from a plurality of digital images stored in a storage device, the metadata including image feature information, geolocation information, and time information, wherein the geolocation information includes distances between a plurality of geographical locations corresponding to the plurality of digital images and the image feature information is based on image characteristics including pixel values of the plurality of digital images;

identifying, by the computing device, a related digital image group that includes a portion of the plurality of digital images from a common geographical area and a common time period, wherein the identifying is based on the metadata including the image feature information that the portion of the plurality of digital images has in common and a clustering algorithm that uses the time information and the geolocation information including the distances between the plurality of geographical locations that correspond to the portion of the plurality of digital images from the common geographical area and the common time period;

grouping, by the computing device, the portion of the plurality of digital images in the related digital image group into a digital image collection based on a semantic relatedness of the image feature information in the portion of the plurality of digital images to a repository of feature information;

identifying, for the digital image collection, by the computing device, an event, wherein the event is identified in a repository of event information based on the common time period and the common geographical area, and wherein the repository of event information comprises a publicly available source of event information;

obtaining a home address of a user; and

assigning, by the computing device, a title to the digital image collection, wherein

in response to the geolocation information matching the home address, the title is based in part on a descriptor of the event and a descriptor of the time information, and

in response to the geolocation information not matching the home address, the title is based in part on a descriptor of the geolocation information, and the descriptor of the time information.

2. The method of claim 1, wherein the repository of event information further comprises non-public information that is associated with the user.

3. The method of claim 1, wherein each of the plurality of digital images is associated with the user and the repository of event information further comprises calendar data associated with the user.

4. The method of claim 1, wherein each of the plurality of digital images is associated with the user and the repository of event information further comprises information obtained from electronic mail messages that are associated with the user.

5. The method of claim 1, wherein each of the plurality of digital images is stored at a server-based storage system and are associated with an account that is associated with the user.

6. The method of claim 1, further comprising:

prior to assigning the title to the digital image collection, suggesting, by a titling component of an application hosting service, the title to a user for acceptance or rejection; and

training the titling component of the application hosting service based on receipt of an acceptance or a rejection.

7. The method of claim 1, wherein the title is further based on a descriptor of a number of digital images in the digital image collection, a descriptor of proximity to a geolocation based on the geolocation information, a descriptor of a date range based on the time information, a descriptor of a person represented in the digital image collection, or a descriptor of a landmark in the digital image collection.

8. A system for organizing digital images by events, the system comprising:

a computing device including a processor, network interface, and a memory, wherein the memory of the computing device includes applications executing instructions, wherein the applications comprise a data extraction component, a grouping component, and a titling component, and wherein the instructions are executable by the processor of the computing device to;

extract, by a data extraction component, metadata from a plurality of digital images stored in a storage device, the metadata including image feature information, geolocation information and time information, wherein the geolocation information includes distances between a plurality of geographical locations corresponding to the plurality of digital images and the image feature information is based on image characteristics including pixel values of the plurality of digital images;

identify, by the grouping component, a related digital image group that includes a portion of the plurality of digital images from a common geographical area and a common time period, wherein the identifying is based on the metadata including the image feature information that the portion of the plurality of digital images has in common and a clustering algorithm that uses the time information and the geolocation information including the distances between the plurality of geographical locations that correspond to the portion of the plurality of digital images from the common geographical area and the common time period;

group, by the grouping component, the portion of the plurality of digital images in the related digital image group into a digital image collection based on a semantic relatedness of the image feature information in the portion of the plurality of digital images to a repository of feature information;

identify, for the digital image collection, by the titling component, an event, wherein the event is identified in a repository of event information based on the common time period and the common geographical area, and wherein the repository of event information comprises a publicly available source of event information;

obtain a home address of a user; and

assign, by the titling component, a title to the digital image collection, wherein

in response to the geolocation information matching the home address, the title is based in part on a descriptor of the event, the image feature information, and a descriptor of the time information, and

in response to the geolocation information not matching the home address, the title is based in part on a descriptor of the geolocation information, the image feature information, and the descriptor of the time information.

9. The system of claim 8, wherein the plurality of digital images is a plurality of unorganized digital images.