CONTENT CLUSTERING SYSTEM AND METHOD

• US 20140280122A1
• Filed: 03/15/2013
• Published: 09/18/2014
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Filmstrip, Inc.
• Original Assignee: Ambient Consulting, LLC
• Inventors: Grossman, Andrew, Krutsch, Kenneth F., Monson-Haefel, Richard
• Granted Patent: US 9,626,365 B2
• Time in Patent Office: Days
• US Class Current: 707/733
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A computerized system and method are presented that creates implicit content on a mobile device by monitoring and recording input from sensors on the device. Metadata from the implicit content and from user-created content is then analyzed the purpose of event identification. Using the metadata and event identification, the content is created into clusters, which can be confirmed by the user as actual events. Events can then be grouped according to metadata and event information into a presentation grouping.

Claims

1. A mobile communication device comprising:

a) a processor that is controlled via programming instructions;

b) a non-transitory computer readable memory;

c) a user input device for receiving explicit input instructions from a user;

d) an optical sensor;

e) non-optical sensors selected from a group consisting of an accelerometer, a gyroscope, and a location identifying sensor;

f) explicit content generation programming stored on the memory and performed by the processor, the explicit content generation programming causing the processor to respond to an explicit input instruction from the user input device by storing image content on the memory, the image content including;

i) an image file recorded by the optical sensor, and

ii) image time metadata indicating the time at which the image file was captured;

g) implicit content generation programming stored on the memory and performed by the processor, the implicit content generation programming causing the processor to;

i) monitor the non-optical sensors;

ii) identify a change in the non-optical sensors;

iii) in response to the change in the non-optical sensors, storing implicit content on the memory, the implicit content including

(1) an indication of the change in the non-optical sensors, and

(2) implicit time metadata identifying the time at which the change in the non-optical sensors was identified;

h) content clustering programming that groups the image content and the implicit content into a cluster based on similarities between the image time metadata and the implicit time metadata.

2. The mobile communication device of claim 1, further comprising

i) calendaring programming that instructs the processor to maintain calendar events on the memory, each calendar event relating to a calendar time and having an event title;

wherein the content clustering program groups at least one calendar event in the cluster based on similarities between the calendar time, the image time metadata and the implicit time metadata.

3. The mobile communication device of claim 1, wherein the content clustering program generates an organized media file on the memory having cluster metadata identifying a time and a location for the cluster, the organized media file identifying the image content and the implicit content as part of the cluster.

4. The mobile communication device of claim 3, further comprising:

i) a network interface allowing data connections with a remote computing device over a wide area network; and

j) remote event querying programming stored on the memory and performed by the processor, the remote event querying programming causing the processor to;

i) query a remote occurrences database over the network interface, the query identifying the cluster time and the cluster location,

ii) receive occurrence data from the remote occurrences database concerning an occurrence that occurred at the cluster time and cluster location;

wherein the occurrence data is added to the cluster metadata.

5. The mobile communication device of claim 4, wherein the memory contains a plurality of image content each having a separate image time metadata, and further wherein the content clustering programming uses the occurrence duration received with the occurrence data to define a time period that determines which of the plurality of image content is included in the content cluster.

6. The mobile communication device of claim 1,

wherein the explicit content generation program creates a plurality of still image files and a plurality of video image files on the memory, wherein each still image file and each video image file is stored with image time metadata indicating the time at which the optical sensor captured the image file;

wherein the implicit content generation programming creates a plurality of implicit content files on the memory, wherein each implicit content file is stored with implicit time metadata identifying the time at which the change in the non-optical sensors was noted that caused the creation of the implicit content file; and

wherein the content clustering program generates an organized media file on the memory, the organized media file identifying a plurality of content clusters, each content cluster consisting of still image files, video image files, and implicit content files.

7. A method for organizing media files on a mobile device comprising:

a) accessing visual media files taken by an image sensor on the mobile device;

b) accessing audio media files taken by a microphone on the mobile device;

c) analyzing, using a processor on the mobile device, time metadata and location metadata for the media files to find commonalities in the metadata;

d) clustering, using the processor, the media files into media clusters without user involvement, the clustering being based both upon commonalities in the time metadata and commonalities in the location metadata.

8. The method of claim 7, further comprising accessing calendar event data maintained by a calendar program on the mobile device, further wherein the clustering step also clusters the calendar event data into the media clusters based on commonalities between the time and location metadata of the media files and time and location information stored in the calendar event data.

9. The method of claim 7, further comprising:

e) creating an organized media file identifying the media clusters.

10. The method of claim 9, wherein the organized media file is stored on remote storage accessible to the processor on the mobile device over a network interface and a wide area network.

11. The method of claim 10, wherein the organized media file includes all of the media content from the media files within the media clusters.

12. The method of claim 9, further comprising:

f) submitting a query to a remote occurrences database over a network interface of the mobile device, the query identifying a cluster time and a cluster location for a particular media cluster; and

g) receiving occurrence data from the remote occurrences database, the occurrences data identifying a occurrence name for an occurrence in the occurrences database that occurred at the cluster time and cluster location;

h) storing the occurrence name as a name for the particular media cluster within the organized media file.

13. The method of claim 12, wherein the occurrence data further includes a duration for the occurrence, wherein the clustering step uses the duration for the occurrence to establish a time frame for media cluster.

14. The method of claim 9, further comprising:

e) providing a user interface on the mobile device for a user to combine, divide, and accept the media clusters to create user-accepted events each comprising a plurality of media files.

15. The method of claim 14, further comprising:

f) identifying participants in the user-accepted events by identifying participants in the media files in the user-accepted events;

16. The method of claim 14, further comprising:

g) creating event metadata for user-accepted events, the event metadata including an event title, event participants, event location, and event time.

17. The method of claim 16, wherein the step of creating event metadata further comprises accessing calendar event data maintained by a calendar program on the mobile device to identify a calendar event for the event time and using data associated with the calendar event to create the event metadata.

18. The method of claim 16, wherein the step of creating event metadata further comprises:

i) submitting a query to a remote occurrences database over a network interface of the mobile device, the query identifying a first event time and a first event location; and

ii) receiving occurrence data from the remote occurrences database for an occurrence stored at the remote occurrences database that occurred at the first event time and the first event location; and

iii) using the occurrences data to create event metadata for a first event.

19. The method of claim 18, further comprising:

h) submitting event metadata over to the remote occurrences database to be added to the database.

20. The method of claim 16, further comprising:

h) accessing event metadata for a plurality of user-accepted events;

i) analyzing, using the processor on the mobile device, the event metadata to find commonalities in the event metadata;

j) generating a presentation grouping of a plurality of user-accepted events based upon commonalities discovered in the event metadata.

21. The method of claim 20, wherein the commonalities discovered in the event metadata relate to an event type metadata retrieved from querying a remote occurrences database.

22. The method of claim 20, wherein the commonalities discovered in the event metadata relate to similarities in textual location descriptions in the metadata.

23. The method of claim 20, wherein the commonalities discovered in the event metadata relate to event participants metadata.

24. The method of claim 20, wherein the commonalities discovered in the event metadata relate to metadata that originated in calendar event data.

25. The method of claim 7, further comprising:

e) generating implicit content files by;

i) monitoring a non-optical sensor on the mobile device,

ii) identifying a change in the non-optical sensors, and

iii) in response to identifying the change, generating an implicit content file containing an identification of the identified change and also containing implicit metadata identifying the time and location at which the change in the non-optical sensors was identified;

wherein the clustering step further clusters the media files with the implicit content files in the media clusters based on commonalities between the implicit metadata and the time metadata and location metadata of the media files.