Method and apparatus for generating video descriptions

• US 9,191,639 B2
• Filed: 05/28/2010
• Issued: 11/17/2015
• Est. Priority Date: 04/12/2010
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Kuspa, David A.
• Primary Examiner(s): Baderman, Scott
• Assistant Examiner(s): Cortes, Howard
• Application Number:
  US12/789,760
• Publication Number:
  US 20130120654A1
• Time in Patent Office: 1,999 Days
• Field of Search: 715/201, 715/203
• US Class Current: 1/1
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 25/57 for processing of video signals
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/440236 by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
• CPC Class Codes: H04N 21/47205 for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content
• CPC Class Codes: H04N 5/445 for displaying additional information H04N5/50 takes precedence
• CPC Class Codes: H04N 9/475 for mutually locking different synchronisation sources

Abstract

Provided in some embodiments is a computer implemented method that includes receiving time-aligned script data including dialogue words of a script and timecodes corresponding to the dialogue words, identifying gaps between dialogue words for the insertion of video description content, wherein the gaps are identified based on the duration of pauses between timecodes of adjacent dialogue words, aligning segments of video description content with corresponding gaps in dialogue, wherein the video description content for the segments is derived from corresponding script elements of the script; and generating a script document including the aligned segments of video description content.

Claims

1. A method, comprising:

receiving time-aligned script data comprising dialogue words of a script and timecodes corresponding to the dialogue words;

comparing pauses between timecodes of adjacent dialogue words to a gap threshold that corresponds to a pre-specified period of time;

ignoring pauses that do not meet or exceed the gap threshold;

identifying pauses that meet or exceed the gap threshold as gaps between dialogue words for the insertion of video description content;

aligning segments of video description content with corresponding gaps in dialogue, wherein the video description content for the segments is derived from corresponding script elements of the script;

comparing a duration of a gap to a length of corresponding video description content to determine if the gap is of sufficient length for the insertion of the corresponding video description content;

in an event that the gap is not of sufficient length, processing, responsive to determining that the gap is not of sufficient length and independent of user input, one or more of the gap length or the corresponding video description content such that the corresponding video description content is insertable into the gap; and

generating a script document comprising the aligned segments of video description content.

2. The method of claim 1, wherein aligning segments of video description content with corresponding gaps in dialogue comprises identifying pauses in the dialogue that occur at the same location as the script elements occur relative to the dialogue words.

3. The method of claim 1, wherein processing one or more of the gap length or the corresponding video description content comprises automatically removing a portion of the video description content using rules of grammar such that the video description content fits within the corresponding gap.

4. The method of claim 1, wherein processing one or more of the gap length or the corresponding video description content comprises automatically increasing the gap length such that the video description content fits within the corresponding gap.

5. The method of claim 1, wherein processing one or more of the gap length or the corresponding video description content comprises providing an opportunity for a user to manually modify one or more of the video description content or the gap.

6. The method of claim 1, wherein processing one or more of the gap length or the corresponding video description content comprises inserting at least a portion of the video description content in one or more nearby gaps.

7. The method of claim 1, further comprising for at least one pause that does not meet or exceed the gap threshold, deleting corresponding video description content.

8. The method of claim 1, wherein processing one or more of the gap length or the corresponding video description content comprises incrementally truncating a last word or a last sentence of the video description content using rules of grammar until a remaining portion of the video description content is insertable into the gap.

9. A hardware computer-readable storage medium comprising program instructions stored thereon, wherein the program instructions are executable to cause a computer system to perform a method comprising:

receiving time-aligned script data comprising dialogue words of a script and timecodes corresponding to the dialogue words;

comparing pauses between timecodes of adjacent dialogue words to a gap threshold that corresponds to a pre-specified period of time;

ignoring pauses that do not meet or exceed the gap threshold;

identifying pauses that meet or exceed the gap threshold as gaps between dialogue words for the insertion of video description content;

aligning segments of video description content with corresponding gaps in dialogue, wherein the video description content for the segments is derived from corresponding script elements of the script;

comparing a duration of a gap to a length of corresponding video description content to determine if the gap is of sufficient length for the insertion of the corresponding video description content;

in an event that the gap is not of sufficient length, processing, responsive to determining that the gap is not of sufficient length and independent of user input, one or more of the gap length or the corresponding video description content such that the corresponding video description content is insertable into the gap; and

generating a script document comprising the aligned segments of video description content.

10. The storage medium of claim 9, wherein aligning segments of video description content with corresponding gaps in dialogue comprises identifying pauses in the dialogue that occur at the same location as the script elements occur relative to the dialogue words.

11. The storage medium of claim 9, wherein processing one or more of the gap length or the corresponding video description content comprises automatically removing a portion of the video description content such that the video description content fits within the corresponding gap.

12. The storage medium of claim 9, wherein processing one or more of the gap length or the corresponding video description content comprises automatically increasing the gap length such that the video description content fits within the corresponding gap.

13. The storage medium of claim 9, wherein processing one or more of the gap length or the corresponding video description content comprises providing an opportunity for a user to manually modify one or more of the video description content or the gap.

14. The storage medium of claim 9, wherein processing one or more of the gap length or the corresponding video description content comprises inserting at least a portion of the video description content in one or more nearby gaps.

15. The computer-readable storage medium of claim 9, wherein the method further comprises for at least one pause that does not meet or exceed the gap threshold, deleting corresponding video description content.

16. A hardware computer system configured to perform operations comprising:

receiving time-aligned script data comprising dialogue words of a script and timecodes corresponding to the dialogue words;

comparing pauses between timecodes of adjacent dialogue words to a gap threshold that corresponds to a pre-specified period of time;

ignoring pauses that do not meet or exceed the gap threshold;

identifying pauses that meet or exceed the gap threshold as gaps between dialogue words for the insertion of video description content;

aligning segments of video description content with corresponding gaps in dialogue, wherein the video description content for the segments is derived from corresponding script elements of the script;

comparing a duration of a gap to a length of corresponding video description content to determine if the gap is of sufficient length for the insertion of the corresponding video description content;

in an event that the gap is not of sufficient length, processing, responsive to determining that the gap is not of sufficient length and independent of user input, one or more of the gap length or the corresponding video description content such that the corresponding video description content is insertable into the gap; and

generating a script document comprising the aligned segments of video description content.

17. The system of claim 16, wherein aligning segments of video description content with corresponding gaps in dialogue comprises identifying pauses in the dialogue that occur at the same location as the script elements occur relative to the dialogue words.

18. The system of claim 16, wherein processing one or more of the gap length or the corresponding video description content comprises automatically removing a portion of the video description content such that the video description content fits within the corresponding gap.

19. The system of claim 16, wherein processing one or more of the gap length or the corresponding video description content comprises automatically increasing the gap length such that the video description content fits within the corresponding gap.

20. The system of claim 16, wherein processing one or more of the gap length or the corresponding video description content comprises providing an opportunity for a user to manually modify one or more of the video description content or the gap.

21. The system of claim 16, wherein processing one or more of the gap length or the corresponding video description content comprises inserting at least a portion of the video description content in one or more nearby gaps.