Grouping digital media items based on shared features

• US 8,611,678 B2
• Filed: 09/27/2010
• Issued: 12/17/2013
• Est. Priority Date: 03/25/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Hanson, Eric, Fagans, Joshua David, Shah, Hemal
• Primary Examiner(s): Koziol, Stephen R
• Assistant Examiner(s): CONNER, SEAN M
• Application Number:
  US12/891,252
• Publication Number:
  US 20110235858A1
• Time in Patent Office: 1,177 Days
• Field of Search: 382/155, 382159-161, 382225-228
• US Class Current: 382/225
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 18/23 Clustering techniques

Abstract

Methods, apparatuses, and systems for grouping digital media items based on shared features. Multiple digital images are received. Metadata about the digital images is obtained either by analyzing the digital images or by receiving metadata from a source separate from the digital images or both. The obtained metadata is analyzed by data processing apparatus to identify a common feature among two or more of the digital images. A grouping of the two or more images is formed by the data processing apparatus based on the identified common feature.

Claims

1. A computer-implemented method comprising:

receiving a plurality of digital images, each digital image including a plurality of faces of people, wherein a first face of a first person and a second face of a second person are common to at least two digital images of the plurality of digital images;

obtaining an identity of the first person;

determining, by data processing apparatus, a frequency of appearances of the first person and the second person in the plurality of digital images;

deriving a relationship between the first person and the second person based on the frequency of appearances of the first person and the second person in the plurality of digital images and the identity of the first person;

forming, by the data processing apparatus, a grouping of the at least two digital images that include the first face of the first person and the second face of the second person into a cluster, wherein features associated with the plurality of digital images include metadata associated with the digital images, wherein the at least two digital images grouped to form the cluster share at least a portion of the metadata, wherein the metadata includes times of capture and geographic locations of capture of the plurality of the digital images, wherein the operations for identifying each of the at least two digital images that share the feature further include;

identifying digital images having times of capture within a duration of time; and

forming a first cluster of the digital images having the times of capture within the duration;

identifying, from among the first cluster, digital images having geographic locations of capture within a region; and

forming a second cluster of the digital images within the first cluster, wherein the digital images in the second cluster having the geographic locations of capture within the region;

deriving one or more keywords describing the two or more images by analyzing content of each of the two or more images;

identifying a theme for the two or more images in the grouping based at least in part on the keywords and at least in part on the relationship between the first person and the second person; and

creating a digital composite media presentation of the at least two digital images according to the themeby;

including the digital images in the first cluster in a first sequence in the digital media presentation, and

including the digital images in the second cluster in a second sequence within the first sequence in the digital media presentation; and

presenting the digital composite media presentation.

2. The method of claim 1, wherein creating the digital composite media presentation of the at least two digital images comprises presenting the at least two digital images concurrently according to the theme.

3. The method of claim 2, wherein the determining, the deriving, the forming, and the creating are performed automatically without input.

4. The method of claim 1, further comprising obtaining image information about the plurality of digital images by analyzing the plurality of digital images, wherein for each digital image, the image information includes pixel information identifying a content captured in the digital image.

5. The method of claim 1, wherein deriving the relationship between the first person and the second person and grouping the at least two digital images that include the first face of the first person and the second face of the second person into the cluster comprises:

analyzing image information about a first digital image to determine that content in the first digital image includes a plurality of faces including the first face;

analyzing image information about a second digital image to determine that content in the second digital image includes the second face included in the plurality of faces in the first digital image; and

forming a grouping of the first digital image and the second digital image in response to determining that content in the second digital image includes the second face included in the plurality of faces in the first digital image.

6. The method of claim 5, further comprising:

presenting the first digital image; and

presenting an animation, the animation including zooming in on the face included in the second digital image that is also included in the plurality of faces in the first digital image in real time.

11. The method of claim 1, wherein deriving one or more keywords describing the two or more images by analyzing content of each of the two or more images comprises:

inferring data describing the two or more images in the grouping based at least in part on content of the two or more digital images and the common feature, wherein the inferred data is different from the metadata; and

forming a set of semantically-significant keywords about the two or more digital images in the grouping.

12. The method of claim 11, wherein the two or more images are arranged in a hierarchy, and wherein inferring the data describing the two or more images in the grouping comprises:

analyzing low-level image features of the two or more images; and

correlating the low-level image features into higher-level features.

7. A non-transitory computer-readable medium tangibly encoding computer software instructions executable by data processing apparatus to perform operations comprising:

receiving a plurality of digital images, each digital image including a plurality of faces of people, wherein a first face of a first person and a second face of a second person are common to at least two digital images of the plurality of digital images;

receiving an identity of the first person;

identifying a frequency of appearances of the first person and the second person in the plurality of digital images;

deriving a relationship between the first person and the second person based on the frequency of appearances of the first person and the second person in the plurality of digital images and the identity of the first person;

grouping the at least two digital images that include the first face of the first person and the second face of the second person into a cluster, wherein features associated with the plurality of digital images include metadata associated with the digital images, wherein the at least two digital images grouped to form the cluster share at least a portion of the metadata, wherein the metadata includes times of capture and geographic locations of capture of the plurality of the digital images, wherein the operations for identifying each of the at least two digital images that share the feature further include;

identifying digital images having times of capture within a duration of time; and

forming a first cluster of the digital images having the times of capture within the duration;

identifying, from among the first cluster, digital images having geographic locations of capture within a region; and

forming a second cluster of the digital images within the first cluster, wherein the digital images in the second cluster having the geographic locations of capture within the region;

identifying a theme associated with the two or more digital images based at least in part on the relationship between the first person and the second person;

creating a digital composite media presentation that includes the cluster according to the theme by;

including the digital images in the first cluster in a first sequence in the digital media presentation, and

including the digital images in the second cluster in a second sequence within the first sequence in the digital media presentation; and

presenting the digital composite media presentation.

8. The medium of claim 7, wherein identifying the theme comprises selecting a presentation template from a plurality of presentation templates, each presentation template associated with a respective theme, the selected presentation template associated with the identified theme, and wherein the cluster is included in the digital composite media presentation according to the selected presentation template.

9. The medium of claim 7, wherein the plurality of digital images are received as a sequence, and wherein the digital images included in the cluster are out of sequence.

10. The medium of claim 7, wherein the features associated with the plurality of digital images include pixel information representing faces of persons, the operations further comprising:

analyzing a first digital image to identify the first face of the first person;

analyzing a second digital image to identify the second face of the second person;

determining that the second face included in the second digital image is included among the plurality of faces in the first digital image; and

grouping the first digital image and the second digital image to form the cluster of related digital images in response to the determining.

13. The medium of claim 7, the operations further comprising:

inferring data describing the at least two digital images based at least in part on content of the at least two digital images; and

forming a set of semantically-significant keywords about the at least two digital images.

14. The medium of claim 7, wherein the at least two digital images are arranged in a hierarchy, and wherein inferring the data that is shared by the at least two digital images comprises:

analyzing low-level image features of the at least two digital images; and

correlating the low-level image features into higher-level features.

15. A system comprising:

data processing apparatus; and

a non-transitory computer-readable medium tangibly encoding computer software instructions executable by the data processing apparatus to perform operations comprising;

receiving a plurality of digital images, each digital image including a plurality of faces of people, wherein a first face of a first person and a second face of a second person are common to at least two digital images of the plurality of digital images;

receiving an identity of the first person;

identifying a frequency of appearances of the first person and the second person in the plurality of digital images;

deriving a relationship between the first person and the second person based on the frequency of appearances of the first person and the second person in the plurality of digital images and the identity of the first person;

grouping the at least two digital images that include the first face of the first person and the second face of the second person into a cluster, wherein features associated with the plurality of digital images include metadata associated with the digital images, wherein the at least two digital images grouped to form the cluster share at least a portion of the metadata, wherein the metadata includes times of capture and geographic locations of capture of the plurality of the digital images, wherein the operations for identifying each of the at least two digital images that share the feature further include;

identifying digital images having times of capture within a duration of time; and

forming a first cluster of the digital images having the times of capture within the duration;

identifying, from among the first cluster, digital images having geographic locations of capture within a region; and

forming a second cluster of the digital images within the first cluster, wherein the digital images in the second cluster having the geographic locations of capture within the region;

identifying a theme associated with the two or more digital images based at least in part on the relationship between the first person and the second person;

creating a digital composite media presentation that includes the cluster according to the theme by;

including the digital images in the first cluster in a first sequence in the digital media presentation, and

including the digital images in the second cluster in a second sequence within the first sequence in the digital media presentation; and

presenting the digital composite media presentation.

16. The system of claim 15, wherein identifying the theme comprises selecting a presentation template from a plurality of presentation templates, each presentation template associated with a respective theme, the selected presentation template associated with the identified theme, and wherein the cluster is included in the digital composite media presentation according to the selected presentation template.

17. The system of claim 15, wherein the plurality of digital images are received as a sequence, and wherein the digital images included in the cluster are out of sequence.

18. The system of claim 15, wherein the features associated with the plurality of digital images include pixel information representing faces of persons, the operations further comprising:

analyzing a first digital image to identify the first face of the first person;

analyzing a second digital image to identify the second face of the second person;

determining that the second face included in the second digital image is included among the plurality of faces in the first digital image; and

grouping the first digital image and the second digital image to form the cluster of related digital images in response to the determining.

19. The system of claim 15, the operations further comprising:

inferring data describing the at least two digital images based at least in part on content of the at least two digital images; and

forming a set of semantically-significant keywords about the at least two digital images.

20. The system of claim 19, wherein the at least two digital images are arranged in a hierarchy, and wherein inferring the data that is shared by the at least two digital images comprises:

analyzing low-level image features of the at least two digital images; and

correlating the low-level image features into higher-level features.