Content clustering system and method

• US 9,626,365 B2
• Filed: 03/15/2013
• Issued: 04/18/2017
• Est. Priority Date: 03/15/2013
• Status: Expired due to Fees

• Current Assignee: Filmstrip, Inc.
• Original Assignee: Ambient Consulting, LLC
• Inventors: Grossman, Andrew, Krutsch, Kenneth F., Monson-Haefel, Richard
• Primary Examiner(s): Corrielus, Jean M
• Application Number:
  US13/832,177
• Publication Number:
  US 20140280122A1
• Time in Patent Office: 1,495 Days
• Field of Search: 707733, 715771, 715243
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A computerized system and method are presented that creates implicit content on a mobile device by monitoring and recording input from sensors on the device. Metadata from the implicit content and from user-created content is then analyzed the purpose of event identification. Using the metadata and event identification, the content is created into clusters, which can be confirmed by the user as actual events. Events can then be grouped according to metadata and event information into a presentation grouping.

Claims

1. A mobile communication device comprising:

a) a processor that is controlled via programming instructions;

b) a non-transitory computer readable memory;

c) a user input device for receiving explicit input instructions from a user;

d) a camera sensor;

e) a non-camera sensor selected from a group consisting of an accelerometer, a gyroscope, and a location identifying sensor;

f) explicit content generation programming stored on the memory and performed by the processor, the explicit content generation programming causing the processor to respond to an explicit input instruction from the user input device by storing image content on the memory, the image content including;

i) an image file recorded by the camera sensor, and

ii) image time metadata indicating the time at which the image file was captured;

g) implicit content generation programming stored on the memory and performed by the processor, the implicit content generation programming causing the processor to;

iii) monitor the non-camera sensor;

iv) identify a change in the non-camera sensor;

v) in response to the change in the non-camera sensor, storing implicit content on the memory, the implicit content including

(1) an indication of the change in the non-camera sensor, and

(2) implicit time metadata identifying the time at which the change in the non-camera sensor was identified;

h) after steps f) and g), content clustering programming that groups the previously created image content and the previously created implicit content into a cluster based on similarities between the image time metadata and the implicit time metadata;

i) a network interface allowing data connections with a remote computing device over a wide area network; and

j) remote event querying programming stored on the memory and performed by the processor, the remote event querying programming causing the processor to;

vi) query a remote occurrences database over the network interface, the query identifying the cluster time and the cluster location,

vii) receive occurrence data from the remote occurrences database concerning an occurrence that occurred at the cluster time and cluster location;

wherein the occurrence data is added to the cluster metadata;

wherein the content clustering program generates an organized media file on the memory having cluster metadata identifying a time and a location for the cluster, the organized media file identifying the image content and the implicit content as part of the cluster; and

wherein the memory contains a plurality of image content each having a separate image time metadata, and further wherein the content clustering programming uses the occurrence duration received with the occurrence data to define a time period that determines which of the plurality of image content is included in the content cluster.

2. A mobile communication device comprising:

a) a processor that is controlled via programming instructions;

b) a non-transitory computer readable memory;

c) a user input device for receiving explicit input instructions from a user;

d) a camera sensor;

e) a non-camera sensor selected from a group consisting of an accelerometer, a gyroscope, and a location identifying sensor;

f) explicit content generation programming stored on the memory and performed by the processor, the explicit content generation programming causing the processor to respond to an explicit input instruction from the user input device by storing image content on the memory, the image content including;

i) an image file recorded by the camera sensor, and

ii) image time metadata indicating the time at which the image file was captured;

g) implicit content generation programming stored on the memory and performed by the processor, the implicit content generation programming causing the processor to;

iii) monitor the non-camera sensor;

iv) identify a change in the non-camera sensor;

v) in response to the change in the non-camera sensor, storing implicit content on the memory, the implicit content including

(1) an indication of the change in the non-camera sensor, and

(2) implicit time metadata identifying the time at which the change in the non-camera sensor was identified;

h) after steps f) and g), content clustering programming that groups the previously created image content and the previously created implicit content into a cluster based on similarities between the image time metadata and the implicit time metadata;

wherein the explicit content generation program creates a plurality of still image files and a plurality of video image files on the memory, wherein each still image file and each video image file is stored with image time metadata indicating the time at which the camera sensor captured the image file;

wherein the implicit content generation programming creates a plurality of implicit content files on the memory, wherein each implicit content file is stored with implicit time metadata identifying the time at which the change in the non-camera sensors was noted that caused the creation of the implicit content file; and

wherein the content clustering program generates an organized media file on the memory, the organized media file identifying a plurality of content clusters, each content cluster consisting of still image files, video image files, and implicit content files.

3. A method for organizing media files on a mobile device comprising:

a) accessing visual media files previously taken by an image sensor on the mobile device and accessing audio media files previously taken by a microphone on the mobile device, wherein the media files each have time metadata and location metadata;

b) analyzing, using a processor on the mobile device, time metadata and location metadata for the media files to find commonalities in the metadata;

c) clustering, using the processor, the media files into media clusters without user involvement, the clustering being based both upon commonalities in the time metadata and commonalities in the location metadata; and

d) establishing for each media cluster, using the processor and without user involvement, cluster metadata, wherein the cluster metadata includes time metadata that specifies a time duration spanning the time metadata of all of the media files in the media cluster, further wherein the cluster metadata includes location metadata based on commonalities in the location metadata for all of the media files in the media cluster; and

f) grouping, using the processor, a plurality of the media clusters into a presentation grouping without user involvement, the presentation grouping being based upon commonalities in the metadata of the plurality of media clusters.

4. The method of claim 3, further comprising accessing calendar event data maintained by a calendar program on the mobile device, further wherein the clustering step also clusters the calendar event data into the media clusters based on commonalities between the time and location metadata of the media files and time and location information stored in the calendar event data, further wherein the commonalities in the metadata used for the grouping of the plurality of media clusters originates in the calendar event data.

5. The method of claim 3, wherein the presentation grouping and the plurality of media clusters are stored on remote storage accessible to the processor on the mobile device over a network interface and a wide area network.

6. The method of claim 5, wherein all of the media content from the media files within the plurality of media clusters is stored on the remote storage.

7. The method of claim 3, further comprising:

g) submitting a query to a remote occurrences database over a network interface of the mobile device, the query identifying a cluster time and a cluster location for a particular media cluster; and

h) receiving occurrence data from the remote occurrences database, the occurrences data identifying a occurrence name for an occurrence in the occurrences database that occurred at the cluster time and cluster location;

i) storing the occurrence name as a name for the particular media cluster within the presentation grouping.

8. The method of claim 7, wherein the occurrence data further includes a duration for the occurrence, wherein the clustering step uses the duration for the occurrence to establish a time frame for media cluster.

9. The method of claim 3, further comprising:

g) providing a user interface on the mobile device for a user to combine, divide, and accept the media clusters to create user-accepted events each comprising a plurality of media files, further wherein each user-accepted event contains event metadata that is used to group the user-accepted events into the presentation grouping.

10. The method of claim 9, further comprising:

h) identifying participants in the user-accepted events by identifying participants in the media files in the user-accepted events.

11. The method of claim 9, wherein the event metadata comprises an event title, event participants, event location, and event time.

12. The method of claim 11, wherein the step of creating event metadata further comprises accessing calendar event data maintained by a calendar program on the mobile device to identify a calendar event for the event time and using data associated with the calendar event to create the event metadata.

13. The method of claim 11, wherein the step of creating event metadata further comprises:

i) submitting a query to a remote occurrences database over a network interface of the mobile device, the query identifying a first event time and a first event location; and

ii) receiving occurrence data from the remote occurrences database for an occurrence stored at the remote occurrences database that occurred at the first event time and the first event location; and

iii) using the occurrences data to create event metadata for a first event.

14. The method of claim 13, further comprising:

h) submitting event metadata over to the remote occurrences database to be added to the database.

15. The method of claim 11, further comprising:

h) accessing event metadata for a plurality of user-accepted events;

i) analyzing, using the processor on the mobile device, the event metadata to find commonalities in the event metadata;

j) generating a presentation grouping of a plurality of user-accepted events based upon commonalities discovered in the event metadata.

16. The method of claim 15, wherein the commonalities discovered in the event metadata relate to an event type metadata retrieved from querying a remote occurrences database.

17. The method of claim 15, wherein the commonalities discovered in the event metadata relate to similarities in textual location descriptions in the metadata.

18. The method of claim 15, wherein the commonalities discovered in the event metadata relate to event participants metadata.

19. The method of claim 15, wherein the commonalities discovered in the event metadata relate to metadata that originated in calendar event data.

20. The method of claim 3, further comprising:

e) generating implicit content files by;

i) monitoring a non-camera sensor on the mobile device,

ii) identifying a change in the non-camera sensors, and

iii) in response to identifying the change, generating an implicit content file containing an identification of the identified change and also containing implicit metadata identifying the time and location at which the change in the non-camera sensors was identified;

wherein the clustering step further clusters the media files with the implicit content files in the media clusters based on commonalities between the implicit metadata and the time metadata and location metadata of the media files.