Correlation of recorded video presentations and associated slides

• US 10,665,267 B2
• Filed: 05/28/2019
• Issued: 05/26/2020
• Est. Priority Date: 03/01/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Klinger, Doina L., Nice, David J., Quaggin-Mitchell, Rebecca M., Xu, Fenglian
• Primary Examiner(s): Tran, Thai Q
• Assistant Examiner(s): Park, Sunghyoun
• Application Number:
  US16/423,475
• Publication Number:
  US 20190279685A1
• Time in Patent Office: 364 Days
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: G11B 27/102 Programmed access in sequence to addressed parts of tracks of operating record carriers access by moving the head G11B3/08, G11B5/54, G11B7/085, G11B21/022; by moving the record carrier G11B15/005, G11B17/005, by driving of both record carrier and head G11B15/1816
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: G11B 27/32 on separate auxiliary tracks of the same or an auxiliary record carrier
• CPC Class Codes: G11B 27/326 used signal is a video-frame or a video-field (P.I.P.)
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

Techniques are disclosed for performing a computer-implemented processing of slide presentation videos to automatically generate index locations corresponding to particular slides within a slide presentation video. In embodiments, a slide presentation video is uploaded to a video processing system. The video processing system performs an image analysis to identify each slide within the slide presentation and determine a time window for each occurrence of each slide. An audio analysis is performed to adjust the time window to the start of a sentence that precedes the introduction of the slide. A user interface includes one or more selectable links associated with each slide that link to a corresponding location within the slide presentation video. Similarly, a processed slide presentation video includes selectable links to index to the corresponding slide of the presentation.

Claims

1. A computer-implemented method for creating a linked index for a slide presentation video, comprising:

detecting a plurality of slides in a slide presentation video, wherein each slide is detected as a separate image from another;

generating a list of slides;

determining a list of time windows for each slide, wherein a time window is a duration between a time a slide is introduced and a time the slide is removed from the slide presentation video;

determining a relevance score for each time window;

creating a list including a start time and an end time for each sentence; and

for each time window with the relevance score exceeding a predetermined threshold, generating a link to a position in the slide presentation video corresponding to the start time for a sentence that spans a start of the time window of a corresponding slide, wherein the link is associated with a slide from the list of slides.

2. The method of claim 1, further comprising:

performing a speech-to-text analysis on an audio track of the slide presentation video;

performing a sentence analysis process on text generated from the speech-to-text analysis; and

using a result of the sentence analysis process in creating the list including the start time and the end time for each sentence.

3. The method of claim 2, further comprising discarding a time window in response to a duration of the time window being less than a predetermined threshold.

4. The method of claim 1, further comprising:

performing a silence gap identification process on an audio track of the slide presentation video;

identifying a boundary waveform based on a silence gap identified from the silence gap identification process; and

wherein generating the link to the position in the slide presentation video comprises generating a link to a position corresponding to a start time for a boundary waveform that spans a start of the time window for at least one slide.

5. The method of claim 1, wherein the generating the link includes generating a uniform resource locator including a time index into the slide presentation video.

6. The method of claim 1, further comprising presenting a selectable link on the slide presentation video for at least one slide shown in the slide presentation video, wherein the selectable link accesses a corresponding slide for the slide presentation video.

7. The method of claim 1, further comprising creating a secondary video file for each time window with a relevance score exceeding a predetermined threshold, wherein the secondary video file comprises content of the slide presentation video during a corresponding time window.

8. An electronic device comprising:

a processor;

a memory coupled to the processor, the memory containing instructions, that when executed by the processor, perform the steps of;

detecting a plurality of slides in a slide presentation video, wherein each slide is detected as a separate image from another;

generating a list of slides;

determining a list of time windows for each slide, wherein a time window is a duration between a time a slide is introduced and a time the slide is removed from the slide presentation video;

determining a relevance score for each time window;

creating a list including a start time and an end time for each sentence; and

for each time window with the relevance score exceeding a predetermined threshold, generating a link to a position in the slide presentation video corresponding to the start time for a sentence that spans a start of the time window of a corresponding slide, wherein the link is associated with a slide from the list of slides.

9. The electronic device of claim 8, wherein the memory further contains instructions, that when executed by the processor, perform the steps of:

performing a speech-to-text analysis on an audio track of the slide presentation video;

performing a sentence analysis process on text generated from the speech-to-text analysis; and

using a result of the sentence analysis process in creating the list including the start time and the end time for each sentence.

10. The electronic device of claim 8, wherein the memory further contains instructions, that when executed by the processor, perform the step of discarding a time window in response to a duration of the time window being less than a predetermined threshold.

11. The electronic device of claim 8, wherein the memory further contains instructions, that when executed by the processor, perform the steps of:

analyzing an audio track of the slide presentation video to perform a silence gap identification process;

identifying a boundary waveform, based on a silence gap identified from the silence gap identification process; and

wherein generating a link to a position in the slide presentation video comprises generating a link to a position corresponding to the start time for a boundary waveform that spans a start of the time window for at least one slide.

12. The electronic device of claim 8, wherein the memory further contains instructions, that when executed by the processor, perform the step of generating a uniform resource locator including a time index into the slide presentation video.

13. The electronic device of claim 8, wherein the memory further contains instructions, that when executed by the processor, perform the step of presenting a selectable link on the slide presentation video for at least one slide shown in the slide presentation video, wherein the selectable link accesses a corresponding slide for the slide presentation video.

14. The electronic device of claim 8, wherein the memory further contains instructions, that when executed by the processor, perform the step of creating a secondary video file for each time window with a relevance score exceeding a predetermined threshold, wherein the secondary video file comprises content of the slide presentation video during the corresponding time window.

15. A computer program product for creating a linked index for a slide presentation video on an electronic computing device, the electronic computing device comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the electronic computing device to:

detect a plurality of slides in a slide presentation video, wherein each slide is detected as a separate image from another;

generate a list of slides;

determine a list of time windows for each slide, wherein a time window is a duration between a time a slide is introduced and a time the slide is removed from the slide presentation video;

determine a relevance score for each time window;

create a list including a start time and an end time for each sentence; and

for each time window with the relevance score exceeding a predetermined threshold, generate a link to a position in the slide presentation video corresponding to the start time for a sentence that spans a start of the time window of a corresponding slide, wherein the link is associated with a slide from the list of slides.

16. The computer program product of claim 15, further comprising program instructions executable by the processor to cause the electronic computing device to:

perform a speech-to-text analysis on an audio track of the slide presentation video;

perform a sentence analysis process on text generated from the speech-to-text analysis; and

use a result of the sentence analysis process in creating the list including the start time and the end time for each sentence.

17. The computer program product of claim 16, further comprising program instructions executable by the processor to cause the electronic computing device to discard a time window in response to a duration of the time window being less than a predetermined threshold.

18. The computer program product of claim 16, further comprising program instructions executable by the processor to cause the electronic computing device to:

analyze an audio track of the slide presentation video to perform a silence gap identification process;

identify a boundary waveform, based on a silence gap identified from the silence gap identification process; and

generate a link to a position in the slide presentation video that references a position corresponding to the start time for a boundary waveform that spans a start of a time window for at least one slide.

19. The computer program product of claim 15, further comprising program instructions executable by the processor to cause the electronic computing device to generate a uniform resource locator including a time index into the slide presentation video.

20. The computer program product of claim 15, further comprising program instructions executable by the processor to cause the electronic computing device to present a selectable link on the slide presentation video for at least one slide shown in the slide presentation video, wherein the selectable link accesses a corresponding slide for the slide presentation video.