Clustering photographs for display on a digital picture frame

• US 10,430,986 B2
• Filed: 11/28/2017
• Issued: 10/01/2019
• Est. Priority Date: 10/10/2013
• Status: Active Grant

• Current Assignee: Aura Home, Inc. (Pushd, Inc.)
• Original Assignee: Pushd, Inc.
• Inventors: Frieder, Ophir, Chowdhury, Abdur, Jensen, Eric, Cherry, Ben, Sanford, Richard, Kolcz, Alek
• Primary Examiner(s): Nguyen, Dustin
• Application Number:
  US15/824,369
• Publication Number:
  US 20180150986A1
• Time in Patent Office: 672 Days
• CPC Class Codes: G06F 16/55 Clustering; Classification
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 10/763 Non-hierarchical techniques, e.g. based on statistics of modelling distributions
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 20/35 Categorising the entire scene, e.g. birthday party or wedding scene
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: H04N 1/00129 with a display device, e.g. CRT or LCD monitor with a television apparatus H04N1/00283; arrangements for control of display devices G09G

Abstract

A digital picture frame including a camera integrated with the frame, and a network connection module allowing the frame for direct contact and upload of photos from electronic devices or from photo collections of community members. The integrated camera is used to automatically determine an identity of a frame viewer and can capture gesture-based feedback. The displayed photos are automatically shown and/or changed according to the detected viewers. Clustering photos by content is used to improve display and to respond to photo viewer desires. The photos can be filtered and cropped at the receiver side.

Claims

1. A method of displaying photos on a digital picture frame including a digital display mounted within a frame, a camera connected to the frame, and a network connection module, the method comprising:

automatically determining with the camera an identity of a viewer of the digital picture frame;

automatically clustering from a digital photo collection, via automated facial recognition, photos over a multi-year timeframe of a person of interest to the viewer to obtained clustered photos of the person of interest, wherein the clustering comprises;

preparing a plurality of sub-clusters each including digital photos of the person of interest at a similar time and/or age;

extracting photo facial features within each of the plurality of sub-clusters that comprise a predetermined limited change over the timeframe to form a cluster representation of the person of interest;

comparing each cluster representation to further cluster representations of others of the plurality of sub-clusters to determine related clusters of the person of interest over the timeframe as a function of the cluster representation; and

linking or grouping related sub-clusters to form the clustered photos of the person of interest for display; and

automatically displaying on the digital display the clustered photos of the person of interest from the digital photo collection as a function of the clustering.

2. The method of claim 1, further comprising comparing metadata of the photos to identify or confirm the content.

3. The method of claim 1, further comprising filtering clusters of photos as a function of quality and/or content, wherein photos of low quality or photos including predetermined people, activities, and/or locations are removed from the clusters.

4. The method of claim 1, wherein the cluster representation comprises a cluster centroid of averaged photo features of each of the plurality of clusters, and further comprising:

comparing centroids of the other of the plurality of clusters to determine related clusters; and

linking or grouping the related clusters for display as a function of similar centroids.

5. The method of claim 1, further comprising comparing clusters of photos across more than one digital photo collection over a network and sharing related clusters over the network for viewing on the digital picture frame.

6. The method of claim 5, further comprising determining whether a second digital photo collection includes a predetermined threshold of photos of an individual before displaying photos from the second digital photo collection.

7. The method of claim 1, further comprising forming classification models from clustered photos, and using the classification models to place unassigned photos into an existing cluster.

8. The method of claim 1, automatically clustering photos of a digital photo collection of the viewer and of a second digital photo collection of a community member.

9. The method of claim 1, further comprising:

ranking the photos to be circulated in the slideshow; and

automatically determining an order of presentation, a number of display repetitions, and/or a time period of display for each of the photos as function of the ranking.

10. The digital picture frame of claim 1, wherein the digital photo collection is determined from a viewer profile of the viewer and the digital picture frame automatically changes the display as a function of the viewer profile upon detection of the viewer through the camera.