Automated lecture deconstruction

• US 10,140,379 B2
• Filed: 10/27/2014
• Issued: 11/27/2018
• Est. Priority Date: 10/27/2014
• Status: Active Grant

• Current Assignee: Chegg Incorporated
• Original Assignee: Chegg Incorporated
• Inventors: Raskin, Oliver, Murphy, Colin, Le Chevalier, Vincent
• Primary Examiner(s): Gishnock, Nikolai A
• Application Number:
  US14/524,350
• Publication Number:
  US 20160117339A1
• Time in Patent Office: 1,492 Days
• Field of Search: 434322, 434323
• CPC Class Codes: G06F 16/4387 by the use of playlists
• CPC Class Codes: G06F 16/45 Clustering; Classification
• CPC Class Codes: G06F 16/489 using time information
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06Q 10/02 Reservations, e.g. for tickets, services or events
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 10/0633 Workflow analysis
• CPC Class Codes: G06Q 10/067 Enterprise or organisation modelling
• CPC Class Codes: G06Q 10/0833 Tracking
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 10/101 Collaborative creation, e.g. joint development of products or services
• CPC Class Codes: G06Q 10/103 Workflow collaboration or project management
• CPC Class Codes: G06Q 20/1235 with control of digital rights management [DRM]
• CPC Class Codes: G06Q 20/14 specially adapted for billing systems
• CPC Class Codes: G06Q 20/29 characterised by micropayments
• CPC Class Codes: G06Q 2220/10 Usage protection of distributed data files
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 30/0225 : Avoiding frauds
• CPC Class Codes: G06Q 30/0229 : Multi-merchant loyalty card systems
• CPC Class Codes: G06Q 30/0242 : Determining effectiveness of advertisements
• CPC Class Codes: G06Q 30/0256 : User search
• CPC Class Codes: G06Q 30/0261 : based on user location
• CPC Class Codes: G06Q 30/0276 : Advertisement creation
• CPC Class Codes: G06Q 30/0277 : Online advertisement
• CPC Class Codes: G06Q 30/0601 : Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0605 : Supply or demand aggregation
• CPC Class Codes: G06Q 30/0611 : Request for offers or quotes
• CPC Class Codes: G06Q 30/0613 : Third-party assisted
• CPC Class Codes: G06Q 30/0621 : Item configuration or customization
• CPC Class Codes: G06Q 30/0625 : Directed, with specific intent or strategy
• CPC Class Codes: G06Q 30/0629 : for generating comparisons
• CPC Class Codes: G06Q 30/0633 : Lists, e.g. purchase orders, compilation or processing
• CPC Class Codes: G06Q 30/0641 : Shopping interfaces
• CPC Class Codes: G06Q 30/0643 : Graphical representation of items or shoppers
• CPC Class Codes: G06Q 30/08 : Auctions
• CPC Class Codes: G06Q 40/04 : Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
• CPC Class Codes: G06Q 40/12 : Accounting
• CPC Class Codes: G06Q 50/01 : Social networking
• CPC Class Codes: G06Q 50/16 : Real estate
• CPC Class Codes: G06Q 50/18 : Legal services
• CPC Class Codes: G06Q 50/182 : Alternative dispute resolution
• CPC Class Codes: G06Q 50/184 : Intellectual property management
• CPC Class Codes: G06Q 50/22 : Social work or social welfare, e.g. community support activities or counselling services
• CPC Class Codes: G09B 19/00 : Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26
• CPC Class Codes: G09B 23/00 : Models for scientific, medical, or mathematical purposes, e.g. full-sized devices for demonstration purposes in the nature of toys A63H
• CPC Class Codes: G09B 29/007 : using computer methods
• CPC Class Codes: G09B 5/02 : with visual presentation of the material to be studied, e.g. using film strip
• CPC Class Codes: G09B 5/06 : with both visual and audible presentation of the material to be studied
• CPC Class Codes: G09B 7/00 : Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: H04W 12/06 : Authentication
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: H04W 4/021 : Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
• CPC Class Codes: H04W 4/025 : using location based information parameters
• CPC Class Codes: H04W 4/08 : User group management
• CPC Class Codes: H04W 4/14 : Short messaging services, e.g. short message services [SMS] or unstructured supplementary service data [USSD]
• CPC Class Codes: H04W 40/02 : Communication route or path selection, e.g. power-based or shortest path routing
• CPC Class Codes: H04W 40/20 : based on geographic position or location
• CPC Class Codes: H04W 48/10 : using broadcasted information
• CPC Class Codes: H04W 48/16 : Discovering, processing access restriction or access information
• CPC Class Codes: H04W 52/0219 : where the power saving management affects multiple terminals
• CPC Class Codes: H04W 52/0251 : using monitoring of local events, e.g. events related to user activity
• CPC Class Codes: H04W 52/0264 : by selectively disabling software applications
• CPC Class Codes: H04W 52/0296 : switching to a backup power supply
• CPC Class Codes: H04W 64/00 : Locating users or terminals or network equipment for network management purposes, e.g. mobility management
• CPC Class Codes: H04W 72/0473 : the resource being transmission power
• CPC Class Codes: H04W 72/12 : Wireless traffic scheduling
• CPC Class Codes: H04W 72/30 : Resource management for broadcast services
• CPC Class Codes: H04W 8/005 : Discovery of network devices, e.g. terminals
• CPC Class Codes: H04W 8/18 : Processing of user or subscriber data, e.g. subscribed services, user preferences or user profiles; Transfer of user or subscriber data
• CPC Class Codes: H04W 8/22 : Processing or transfer of terminal data, e.g. status or physical capabilities
• CPC Class Codes: H04W 80/06 : Transport layer protocols, e.g. TCP [Transport Control Protocol] over wireless transmission control protocol/Internet protocol [TCP/IP] or user datagram protocol [UDP] H04L69/16
• CPC Class Codes: H04W 84/12 : WLAN [Wireless Local Area Networks]
• CPC Class Codes: H04W 88/182 : Network node acting on behalf of an other network entity, e.g. proxy
• CPC Class Codes: H04W 92/18 : between terminal devices
• CPC Class Codes: Y02D 30/70 : in wireless communication networks
• CPC Class Codes: Y10S 707/922 : Communications

Abstract

An online platform generates a playlist of clips of a lecture accessed by a plurality of users of the online platform. The online platform receives a recording of the lecture, and receives a plurality of events captured during a time period corresponding to the lecture. Each captured event is associated with a time stamp corresponding to a time at which a user performed an activity while listening to the lecture. The online platform clusters the captured events based on the time stamps, and generates one or more clips of the recording of the lecture from the clustered events. The online platform generates a playlist including the clips of the lecture.

Claims

1. A method for generating a playlist of clips of a lecture accessed by a plurality of users of an online platform, the method comprising:

providing an audio or video recording of the lecture for playback at a plurality of user devices each accessing the online platform;

receiving from each of the plurality of user devices, a plurality of events captured by the online platform while the recording of the lecture is played at the user device, each captured event associated with a time stamp corresponding to a time at which a user of the user device performed an activity associated with the online platform;

filtering the captured events correlated from the plurality of user devices based on the types of activities associated with the captured events, wherein the types of activities include passive, active, and recall activities;

clustering the filtered events from the plurality of user devices using a time-based probability analysis;

generating by the online platform, one or more clips of the recording of the lecture based on the clustered events, each clip including a portion of the recording of the lecture, the portion beginning and ending at times within the recording identified based on the time stamps associated with one of the clusters of events; and

generating a playlist including the clips of the lecture.

2. The method of claim 1, wherein generating the one or more clips of the lecture comprises:

ranking the clusters of events based on a number of events in each cluster;

selecting one or more of the clusters of events based on the ranking; and

generating a clip of the lecture corresponding to each of the selected clusters.

3. The method of claim 1, further comprising:

for each of the one or more clips, extracting a key indicator of the lecture from the captured events corresponding to the clip; and

generating a timeline of the lecture based on the time stamps of the captured events corresponding to the one or more clips, the timeline mapping each extracted key indicator to a time within the lecture.

4. The method of claim 3, wherein generating the timeline of the lecture comprises:

associating each key indicator with a time in the lecture based on the time stamps of the captured events associated with a corresponding clip; and

generating the timeline to illustrate the key indicators at the respective associated times in the lecture.

5. The method of claim 1, wherein a plurality of additional captured events are received from one or more other user devices, the additional captured events each associated with an activity performed by a user of the other user device while the user attends a live version of the lecture in person or accesses a live-streamed lecture through the online platform and a time stamp corresponding to a time within the lecture at which the user performed the activity, the method further comprising:

identifying the lecture based on a course registration of the user; and

clustering the additional captured events with the events captured while the recording of the lecture is played at each of the plurality of user devices based on the time stamps.

6. The method of claim 5, further comprising:

mapping the time stamp associated with each event to a time in the recording based on an amount of time elapsed between a start time of the live lecture and the time stamp associated with a respective event.

7. The method of claim 1, wherein generating one or more clips of the lecture comprises:

selecting a time in the recording of the lecture for a beginning of the clip based on a time stamp associated with an early event in a cluster of events.

8. The method of claim 1, wherein an activity performed by a user comprises one of accessing a page of an electronic document distributed by the online platform, taking a note via the online platform, and answering a question on the online platform.

9. The method of claim 1, wherein generating one or more clips of the recording of the lecture based on the clustered events comprises:

ranking the clusters based on a number of events in each cluster; and

selecting one or more of the highest-ranked clusters.

10. The method of claim 1, wherein generating one or more clips of the recording of the lecture based on the clustered events comprises:

ranking the clusters based on a percentage of the users accessing the lecture who performed activities associated with the events in the cluster; and

selecting one or more of the highest-ranked clusters.

11. The method of claim 1, wherein each clip includes a portion of the recording of the lecture beginning and ending at times within the recording, the times determined using the time stamps associated with one of the clusters of events.

12. The method of claim 1, wherein a time stamp associated with an early event in a cluster is used as a beginning time of a clip, and a time stamp associated with a later event in the cluster is used as an ending time of a clip.

13. A non-transitory computer-readable storage medium storing executable computer program instructions for generating a playlist of clips of a lecture accessed by a plurality of student users of an online education platform, the computer program instructions when executed by a processor causing the processor to:

provide an audio or video recording of the lecture for playback at a plurality of student user devices each accessing the online education platform;

receive from each of the plurality of student user devices, a plurality of events captured by the online platform while the recording of the lecture is played at the student user device, each captured event associated with a time stamp corresponding to a time at which a student user of the student user device performed an educational activity associated with the online education platform;

filtering the captured events correlated from the plurality of student user devices based on the types of activities associated with the captured events, wherein the types of activities include passive, active, and recall activities;

normalize the filtered events using the time stamps, the normalizing comprising combining a plurality of events into a combined event and generating a time stamp for the combined event according to the time stamps associated with the combined plurality of events;

cluster the normalized events from the plurality of student user devices using a time-based probability analysis that takes into account the time stamps of the normalized events;

generate, by the online education platform, one or more clips of the recording of the lecture based on the clustered events, each clip including a portion of the recording of the lecture, the portion beginning and ending at times within the recording identified based on the time stamps associated with one of the clusters of events; and

generate a playlist including the clips of the lecture.

14. The non-transitory computer-readable storage medium of claim 13, wherein the computer program instructions causing the processor to generate the one or more clips of the lecture comprise computer program instructions causing the processor to:

rank the clusters of events based on a number of events in each cluster;

select one or more of the clusters of events based on the ranking; and

generate a clip of the lecture corresponding to each of the selected clusters.

15. The non-transitory computer-readable storage medium of claim 13, wherein a plurality of additional captured events are received from one or more other user devices, the additional captured events each associated with an activity performed by a user of the other user device while the user attends a live version of the lecture in person or accesses a live-streamed lecture through the online platform and a time stamp corresponding to a time within the lecture at which the user performed the activity, further comprising computer program instructions that when executed by the processor cause the processor to:

identify the lecture based on a course registration of the user; and

cluster the additional captured events with the events captured while the recording of the lecture is played at each of the plurality of user devices based on the time stamps.

16. The non-transitory computer-readable storage medium of claim 13, further comprising computer program instructions for augmenting the audio or video recording of the lecture with a secondary audio track or a sub-title track.

17. The non-transitory computer-readable storage medium of claim 13, wherein the information about the educational activity associated with the time stamp comprises one or more of:

an identifier of a type of activity, a course document associated with the activity, and content generated during the activity.

18. The non-transitory computer-readable storage medium of claim 13, wherein normalizing the captured events using the time stamps comprises:

combining two events separated by a small time interval; and

selecting a time stamp for the combined event using the time stamps associated with the two events.

19. The non-transitory computer-readable storage medium of claim 18, further comprising computer program instructions for selecting a midpoint between the time stamps associated with the two events as the time stamp for the combined event.

20. The non-transitory computer-readable storage medium of claim 13, wherein each clip includes a portion of the recording of the lecture beginning and ending at midpoints between clusters.