Data processing system for managing activities linked to multimedia content when the multimedia content is changed

• US 10,102,593 B2
• Filed: 03/29/2018
• Issued: 10/16/2018
• Est. Priority Date: 06/10/2016
• Status: Active Grant

• Current Assignee: RALI Solutions LLC
• Original Assignee: Understory, LLC
• Inventors: Benfield, Steve, Aslami, Ahmad, Marshall, James
• Primary Examiner(s): Jacob, Ajith
• Application Number:
  US15/940,601
• Publication Number:
  US 20180218458A1
• Time in Patent Office: 201 Days
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G06F 16/783 using metadata automatically derived from the content
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 50/01 Social networking

Abstract

An activity management system is configured to allow users to access multimedia content where the multimedia content is divided in to segments. While a user is viewing or interacting with the multimedia content, the user can submit one or more activities (e.g., comments, questions, replies, or reactions) using a graphical user interface. The system is operable to electronically link the activity provided by the user with the particular segment of multimedia content in which the activity was captured by the system. The system is also configured to determine a sentiment score for particular multimedia content, one or more segments of the particular multimedia content, a user, a group of users, or an organization. The sentiment score may be related to a general mood (e.g., confused, happy, bored) of the user when viewing one or more segments of particular multimedia content.

Claims

1. An electronic data processing system for electronically managing a conversation that is linked to one or more segments of multimedia content divided into a plurality of segments, and for syncing the activities to edited multimedia content, comprising:

a. one or more processors;

b. memory coupled to the one or more processors;

c. a plurality of electronic data records stored in the memory, each one of the plurality of electronic data records corresponding to an activity submitted by a viewer of a particular multimedia content and comprising an identifier for the particular multimedia content, information associated with the activity, information regarding the segment of the multimedia content the activity is associated with, and one or more data fields selected from a group consisting of;

i. text contained in the activity;

ii. a time stamp for when the activity was initiated;

iii. a time stamp for when the activity was submitted;

iv. a length of the activity;

v. a number of words contained in the activity;

vi. a reaction type;

vii. a reply count;

viii. an activity count;

ix. a question flag;

x. an answer flag;

xi. an activity type;

xii. an identifier for the viewer who submitted the activity; and

xiii. an identifier for a related activity that is linked to the activity;

wherein the one or more processors are configured for;

i. determining a first hash value for one or more frames in each segment of the particular multimedia content;

ii. at least partially in response to receiving notification that the particular multimedia content has been edited, determining a second hash value for one or more frames in each segment of the edited particular multimedia content;

iii. comparing the second hash value to the first hash value; and

iv. identifying one or more portions of the particular multimedia content that has been edited based at least in part on detecting a change between the second hash values and the first hash values.

2. The electronic data processing system of claim 1, wherein:

a. determining a first hash value for one or more frames in each segment of the particular multimedia content further comprises determining a first hash value for each frame of the particular multimedia content; and

b. determining a second hash value for one or more frames in each segment of the edited particular multimedia content further comprises determining a second hash value for each frame of the edited particular multimedia content.

3. The electronic data processing system of claim 1, wherein the one or more processors are further configured for:

a. identifying one or more of the plurality of electronic data records that are associated with segments of the particular multimedia content that has not been edited; and

b. facilitating an association of the identified one or more of the plurality of electronic data records with corresponding unedited segments of the edited particular multimedia content.

4. The electronic data processing system of claim 3, wherein facilitating an association of the identified one or more of the plurality of electronic data records with corresponding unedited segments of the edited particular multimedia content further comprises adding an identifier for the edited particular multimedia content to the identified one or more of the plurality of electronic data records that correspond to unedited segments.

5. The electronic data processing system of claim 1, wherein the one or more processors are further configured to analyze the particular multimedia content to determine a subject matter of at least one frame in each segment of the multimedia content.

6. The electronic data processing system of claim 1, wherein analyzing the particular multimedia content to determine a subject matter of at least one frame in each segment of the multimedia content further comprises one or more steps selected from the group consisting of:

a. analyzing an audio track associated with each segment of the multimedia content to determine the subject matter of the content contained in each respective segment;

b. analyzing one or more frames of video associated with each particular segment to determine the subject matter of the content contained in each respective segment; and

c. analyzing metadata associated with each segment of the multimedia content to determine the subject matter of the content contained in each respective segment.

7. A computer-implemented data processing method for electronically managing a conversation that is linked to one or more segments of multimedia content divided into a plurality of segments, and for syncing the activities to edited multimedia content, comprising:

a. determining first hash values for one or more frames in each segment of the particular multimedia content;

b. at least partially in response to receiving notification that the particular multimedia content has been edited, determining second hash values for one or more frames in each segment of the edited particular multimedia content;

c. comparing the second hash values to the respective first hash values; and

d. identifying one or more segments of the particular multimedia content that have been edited based at least in part on detecting a change between a second hash value and a respective first hash value;

e. identifying one or more of a plurality of activities that are associated with one or more segments of the particular multimedia content that have not been edited; and

f. facilitating an association of the identified activities with corresponding unedited segments of the edited particular multimedia content.

8. The computer-implemented data processing method of claim 7, further comprising:

identifying the one or more segments of the particular multimedia content that have not been edited based at least in part on identifying one or more of the first hash values among the second hash values.

9. The computer-implemented data processing method of claim 8, identifying the one or more segments of the particular multimedia content that have not been edited based at least in part on metadata associated with a first segment of the particular multimedia content having a first frame with a third hash value and metadata associated with a second segment of the edited particular multimedia content having a second frame with the third hash value.

10. The computer-implemented data processing method of claim 7, wherein facilitating the association of the identified activities with corresponding unedited segments of the edited particular multimedia content comprises modifying an electronic record associated with the unedited segments such that the electronic record comprises an identifier associated with the edited particular multimedia content.

11. The computer-implemented data processing method of claim 7, further comprising:

displaying the identified activities with the corresponding unedited segments of the edited particular multimedia content in association with the unedited piece of media content.

12. A computer-implemented data processing method for mapping one or more activities associated with a respective segment of one or more first segments that make up a piece of multimedia content to a corresponding segment of one or more second segments that make up an edited version of the piece of multimedia content, the method comprising:

a. detecting, by one or more computer processors, one or more modifications to a piece of multimedia content, the one or more modifications resulting in the edited version of the piece of multimedia content;

b. comparing, by one or more computer processors, the piece of multimedia content to the edited version of the piece of multimedia content, to identify a first particular segment of the one or more first segments as a second particular segment of the one or more second segments;

c. retrieving, by one or more computer processors, from memory, a first electronic record based at least in part on the first particular segment, the first electronic record comprising a first activity, a multimedia identifier for the piece of multimedia content, and a first segment identifier for the first particular segment; and

d. creating a second electronic record that comprises one or more of the first activity, a second multimedia identifier for the edited version of the piece of multimedia content, and a second segment identifier for the second particular segment, and digitally storing the second electronic record in memory.

13. The computer-implemented data processing method of claim 12, further comprising:

determining, by one or more computer processors, a group of first hash values for each of the one or more first segments;

determining, by one or more computer processors, a group of second hash values for each of the one or more second segments; and

comparing, by one or more computer processors, the piece of multimedia content to the edited version of the piece of multimedia content, by comparing the group of first hash values to the group of second hash values.

14. The computer-implemented data processing method of claim 13, further comprising:

comparing, by one or more computer processors, the group of first hash values to the group of second hash values to identify at least one deleted segment that the one or more first segments comprise and the one or more second segments do not comprise.

15. The computer-implemented data processing method of claim 14, wherein:

the at least one deleted segment is a time segment;

the piece of multimedia content comprises a video; and

the method further comprises, at least partially in response to identifying the at least one deleted segment;

comparing the group of first hash values to the group of second hash values to identify at least one remaining segment that both the one or more first segments and the one or more second segments comprise; and

modifying an electronic record associated with the at least one remaining segment based at least in part on the at least one deleted segment.

16. The computer-implemented data processing method of claim 12, further comprising:

modifying the first electronic record based at least in part on the comparison of the piece of multimedia content to the edited version of the piece of multimedia content such that the modified first electronic record comprises;

the first activity, the second multimedia identifier for the edited version of the piece of multimedia content, and the second segment identifier for the second particular segment; and

digitally storing the modified first electronic record.

17. The computer-implemented data processing method of claim 12, further comprising:

providing, by an activity management system, a first graphical user interface comprising a first segment display, wherein each segment of the first segment display is associated with a respective segment of the edited version of the piece of multimedia content; and

presenting, on the first graphical user interface, a first representation of the first activity on the first segment display in association with a point on the first segment display corresponding to the second particular segment of the edited version of the piece of multimedia content.

18. The computer-implemented data processing method of claim 17, further comprising:

receiving at a first time, via the first graphical user interface, a second activity from a viewer of the edited version of the piece of multimedia content;

electronically linking, by the activity management system, the received second activity to a third segment of the edited version of the piece of multimedia content; and

creating a second electronic record that comprises one or more of the second activity, the second multimedia identifier for the edited version of the piece of multimedia content, a viewer identifier, and a third segment identifier for the third segment, and digitally storing the first electronic record;

presenting, on the first graphical user interface, a second representation of the second activity on the first segment display in association with a point on the first segment display corresponding to the third segment of the edited version of the piece of multimedia content.

19. A computer-implemented data processing method for mapping a plurality of activities associated with a respective segment of one or more first segments that make up a piece of multimedia content to a corresponding segment of one or more second segments that make up an edited version of the piece of multimedia content, the method comprising:

a. detecting, by one or more computer processors, one or more modifications to a piece of multimedia content, the one or more modifications resulting in the edited version of the piece of multimedia content and the piece of multimedia content being associated with an activity map that defines an association between each particular activity of the plurality of activities and the respective segment of the one or more first segments;

b. comparing, by one or more computer processors, the piece of multimedia content to the edited version of the piece of multimedia content, to identify;

i. at least one remaining segment that the one or more first segments and the one or more second segments comprise, the at least one remaining segment being associated with at least one activity; and

ii. at least one deleted segment that the one or more first segments comprise and the one or more second segments do not comprise;

c. electronically modifying, based at least in part on the comparison, the activity map to generate an edited activity map, the edited activity map being configured to indicate an association between the at least one remaining segment, a corresponding segment of the one or more second segments, and the at least one activity associated with the at least one remaining segment; and

d. electronically associating, by one or more processors, the edited activity map with the edited piece of multimedia content in memory.

20. The computer-implemented data processing method of claim 19, wherein the edited activity map does not comprise the at least one deleted segment.

21. The computer-implemented data processing method of claim 19, further comprising:

determining, by one or more computer processors, a group of first hash values for each of the one or more first segments;

determining, by one or more computer processors, a group of second hash values for each of the one or more second segments; and

comparing, by one or more computer processors, the piece of multimedia content to the edited version of the piece of multimedia content, by comparing the group of first hash values to the group of second hash values.

22. The computer-implemented data processing method of claim 19, further comprising:

dividing each of the one or more first segments into a plurality of first sub-segments;

dividing each of the one or more second segments into a plurality of second sub-segments;

determining, by one or more computer processors, a group of first hash values for each of the plurality of first sub-segments;

determining, by one or more computer processors, a group of second hash values for each of the plurality of second sub-segments; and

comparing, by one or more computer processors, the piece of multimedia content to the edited version of the piece of multimedia content, by comparing the group of first hash values to the group of second hash values.

23. The computer-implemented data processing method of claim 22, wherein:

the piece of multimedia content comprises a video; and

each of the plurality of sub-segments comprise a single frame of the video.