Multi-audio add/drop deterministic animation synchronization

• US 20050188297A1
• Filed: 12/17/2004
• Published: 08/25/2005
• Est. Priority Date: 11/01/2001
• Status: Abandoned Application

• Current Assignee: Automatic E-Learning LLC
• Original Assignee: Automatic E-Learning LLC
• Inventors: Beck, Richard T. IV, Isermann, Peter J., Knight, Jeffrey L., Hill, Shane W., Diesel, Michael E.
• Application Number:
  US11/016,552
• Publication Number:
  US 20050188297A1
• Time in Patent Office: Days
• US Class Current: 715/203
• CPC Class Codes: G09B 5/00 Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: G09B 7/07 providing for individual presentation of questions to a plurality of student stations
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/142 Managing session states for stateless protocols; Signalling session states; State transitions; Keeping-state mechanisms
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

Techniques are provided for synchronizing audio and visual content. A multiple audio language product can be produced containing a single video file that is automatically synchronized to whichever audio the viewer selects. The audio streams and video streams are processed into a plurality of segments. If, for example, an audio stream is selected that corresponds to a particular language, which is not the original audio stream that the video was synchronized to, then the duration of each audio segment in the selected stream can be compared with the duration of each segment in the original audio stream. The number of frames in a segment of the video stream can be adjusted based on the comparison. If the playback duration of the selected audio segment is greater than the corresponding original audio segment, one or more frames in the video segment can be repeated. If the playback duration of the selected audio segment is less than the corresponding original audio segment, then one or more frames in the video segment can be dropped. In this way, video can be automatically synchronized, at run-time, to whichever audio the viewer selects.

Claims

1. A system for synchronizing media content comprising:

a media segment having a media duration;

a first audio segment corresponding to the media segment, the first audio segment having a first audio duration;

a second audio segment corresponding to the media segment, the second audio segment having a second audio duration; and

a processor comparing the first audio duration with the second audio duration and adjusting the media duration to substantially equal the second audio duration based on the comparison.

2. A system as in claim 1 wherein the processor comparing the first audio duration with the second audio duration further includes the processor comparing, at run-time, the media segment and first audio segment.

3. A system as in claim 1 wherein the processor comparing the first audio duration with the second audio duration and adjusting the media duration to substantially equal the second audio duration based on the comparison further includes:

a handler, in communication with the processor, responding to a determination that the duration of the second audio segment is greater than the duration of the first audio segment, by directing the processor to add one or more frames in the media segment.

4. A system as in claim 3 further including the processor, in communication with a player, adding one or more frames to the media segment to increase the duration of the media segment.

5. A system as in claim 4 wherein the player, in communication with the processor, adding one or more frames to the media segment to increase the duration of the media segment further includes the player, in communication with the processor, repeating one or more frames of the media segment.

6. A system as in claim 5 wherein the player, in communication with the processor, repeating one or more frames of the media segment further includes the player, in communication with the processor, repeating every Nth frame of the media segment.

7. A system as in claim 6 wherein the player, in communication with the processor, repeating every Nth frame of the media segment further includes:

the player, in communication with the processor, responding to a determination that the second audio duration is approximately ten percent greater than the first audio duration by causing every tenth frame of the media segment to be repeated.

8. A system as in claim 1 wherein the processor comparing the first audio duration with the second audio duration and adjusting the media duration to substantially equal the second audio duration based on the comparison further includes:

a handler, in communication with the processor, responding to a determination that the second audio duration is less than the first audio segment by removing one or more frames from the media segment by directing the processor to remove one or more frames from the media segment.

9. A system as in claim 7 wherein the processor removing one or more frames from the media segment further includes the player, in communication with the processor, causing the media duration to decrease.

10. A system as in claim 7 wherein the processor removing one or more frames to the media segment further includes the player, in communication with the processor, removing one or more frames to the media segment.

11. A system as in claim 7 wherein the processor removing one or more frames from the media segment further includes the player, in communication with the processor, dropping every Nth frame from the media segment.

12. A system as in claim 8 wherein the processor dropping every Nth frame from the media segment further includes:

the player, in communication with the processor, responding to a determination that the duration of the second audio segment is approximately twenty percent greater than the duration of the first audio segment by dropping every twentieth frame of the media segment.

13. A system as in claim 1 wherein the first audio segment is associated with an initial version of audio and the second audio segment is associated with a subsequent version of the audio.

14. A system as in claim 1 wherein the first audio segment is associated with a first language and the second audio segment is associated with a second language.

15. A system as in claim 14 wherein the first audio segment has corresponding text content in the first language, and the second audio segment has corresponding text content in the second language.

16. A system as in claim 15 wherein the text content for the first and second languages correspond to closed-captioning text for a presentation.

17. A system as in claim 16 wherein the presentation is at least one of an e-learning presentation, interactive exercise, video, animation, or movie.

18. A system as in claim 16 wherein the presentation is created using developer tools, which include an electronic table having rows and columns defining cells.

19. A system as in claim 18 wherein the developer tools for creating the presentation further include:

a time-coder in communication with the electronic table;

the time-coder being responsive to a request to assign time-coding information to a respective media stream, audio stream, or text content; and

the electronic table, in communication with the time-coder, storing identifiers that reflect the time-coding information assigned by the time-coder.

20. A system as in claim 19 wherein the time-coding information controls playback duration of the respective media stream, audio stream, or text content in the presentation.

21. A system as in claim 18 wherein the electronic table enables a user to specify electronic content for a presentation.

22. A system as in claim 21 wherein the electronic content for the presentation is specified in the cells of the electronic table.

23. A system as in claim 22 wherein the electronic content includes media content, audio content or text content.

24. A system as in claim 21 wherein the developer tools further include:

a builder engine processing time-codes specified in the electronic table;

the builder engine generating computer readable instructions based on the time-codes; and

the computer readable instructions defining the presentation.

25. A system as in claim 24 wherein the computer readable instructions are stored in an XML file.

26. A system as in claim 24 wherein the computer readable instructions cause the player to create an array referencing information about the electronic content in an array.

27. A system as in claim 26 wherein the array further includes cells that substantially reflect the arrangement of the cells in the electronic table.

28. A system as in claim 1 wherein the processor adjusting the media duration to substantially equal the second audio duration based on the comparison further includes adjust the media duration without modifying any content stored in the media segment.

29. A system as in claim 1 wherein the media duration is the same as the first audio duration before the processor adjusts the media duration to substantially equal the second audio duration.

30. A system as in claim 1 wherein the media duration reflects the first audio duration before the processor adjusts the media duration to substantially equal the second audio duration further includes:

time-codes associated with the media segment and the first audio segment, where the media segment is substantially synchronized with the first audio segment.

31. A system as in claim 1 wherein the media segment is adjusted to substantially equal the second audio segment without any time-code information associated with the second audio segment.

32. A system as in claim 1 further including:

a media stream having a plurality of media segments, where one of the segments is the media segment;

a first audio stream having a plurality of segments, where one of the segments is the first audio segment; and

a second audio stream having a plurality of segments, where one of the segments is the second audio segment.

33. A system as in claim 1 wherein the processor adjusting the media duration to substantially equal the second audio duration based on the comparison further includes the processor automatically adjusting the media duration.

48. A method as in claim 14 wherein defining the media segments and first and second audio segments using the time-codes further includes:

processing the first and second audio streams by inserting markers at each respective segment; and

responding to the markers by firing an event.

49. A method as in claim 48 wherein the markers are used in comparing the duration of the first audio segment with the duration of the second audio segment.

34. A method for synchronizing media and audio comprising:

processing a media segment and a first audio segment, the media segment having a duration that corresponds to the duration of the first audio segment;

comparing the duration of the first audio segment with a duration of a second audio segment; and

causing the duration of media segment and the duration of the second audio segment to correspond by modifying the duration of the media segment based on the comparison.

35. A method as in claim 34 wherein comparing the duration occurs at run-time.

36. A method as in claim 34 wherein modifying the duration of the media segment based on the comparison further includes:

determining that the duration of the second audio segment is greater than the duration of the first audio segment; and

responding to determining that the duration of the second audio segment is greater than the duration of the first audio segment by adding one or more frames to the media segment.

37. A method as in claim 36 wherein adding one or more frames to the media segment further includes increasing the duration of the media segment.

38. A method as in claim 36 wherein adding one or more frames to the media segment further includes copying one or more frames to the media segment.

39. A method as in claim 36 wherein adding one or more frames to the media segment further includes repeating one or more frames of the media segment.

40. A method as in claim 39 wherein repeating one or more frames of the media segment further includes repeating every Nth frame of the media segment.

41. A method as in claim 40 wherein repeating every Nth frame of the media segment further includes:

determining that the duration of the second audio segment is approximately ten percent greater than the duration of the first audio segment; and

repeating every tenth frame of the media segment.

42. A method as in claim 34 wherein modifying the duration of the media segment based on the comparison further includes:

determining that the duration of the second audio segment is less than the first audio segment by removing one or more frames from the media segment; and

responding to determining that the duration of the second audio segment is less than the first audio segment by removing one or more frames from the media segment.

43. A method as in claim 42 wherein removing one or more frames from the media segment further includes decreasing the duration of the media segment.

44. A method as in claim 42 wherein removing one or more frames from the media segment further includes removing one or more frames from the media segment.

45. A method as in claim 42 wherein removing one or more frames from the media segment further includes dropping every Nth frame from the media segment.

46. A method as in claim 45 wherein dropping every Nth frame from the media segment further includes:

determining that the duration of the second audio segment is approximately twenty percent greater than the duration of the first audio segment; and

dropping every twentieth frame of the media segment.

47. A method as in claim 34 further including:

defining the media segment using time-codes, where the media segment reflects a portion of a media stream, the media stream being portioned into segments with time-codes;

defining the first audio segment using time-codes, where the first audio segment reflects a portion of a first audio stream being substantially synchronized to the media stream, the first audio stream being partitioned into segments using time-codes; and

defining the second audio segment using markers, where the second audio segment reflects a portion of a second audio stream corresponding to the media stream and the first audio stream, the second audio stream being segmented using markers.

50. A method as in claim 47 wherein the first audio stream is associated with an initial version of an audio component for the media stream and the second audio stream is associated with a subsequent version of an audio component for the media stream.

51. A method as in claim 47 wherein the first audio stream is associated with a first language and the second audio stream is associated with a second language.

52. A method as in claim 51 wherein the first audio stream has corresponding text content in the first language, and the second audio stream has corresponding text content in the second language.

53. A method as in claim 52 wherein the respective text content for the first and second languages provide closed-captioning text associated with the media stream for a presentation.

54. A method as in claim 53 wherein the presentation is at least one of an e-learning presentation, interactive exercise, video, animation, or movie.

55. A method as in claim 53 wherein at least a portion of the presentation includes a combination of media content selected from a group consisting of:

the media segment, the first audio segment, the text content of the language of the first audio segment, the second audio segment, the text content of the second audio segment.

56. A method as in claim 53 further includes creating the presentation using an electronic table having rows and columns defining cells.

57. A method as in claim 56 wherein creating the presentation using an electronic table further includes specifying, in the electronic table, indicators identifying respective time-codes for the media stream, the text content, and the first audio stream and the second audio streams.

58. A method as in claim 57 wherein specifying, in the electronic table, the indicators further includes:

storing, in one or more arrays, the respective time-codes defining segments of the media stream, segments of the first audio stream and segments of the second audio stream; and

using the respective time-codes stored in the arrays, controlling the duration of the media stream and the second audio streams.

59. A method as in claim 34 wherein the respective duration of the media segment and the first and second audio segments correspond to time-code information used to synchronize the media segment with the first audio segment or second audio segment.

60. A system for synchronizing media and audio comprising:

means for processing a media segment and a first audio segment, the media segment having a duration that corresponds to the duration of the first audio segment;

means for comparing the duration of the first audio segment with a duration of a second audio segment; and

means for causing the duration of media segment and the duration of the second audio segment to correspond by modifying the duration of the media segment based on the comparison.

61. A system for synchronizing media content comprising:

a media stream having a plurality of media segments, each media segment having a respective media duration;

a first audio having a plurality of first audio segments, each of the first audio segments having a respective first audio duration;

a second audio having a plurality of second audio segments, each of the second audio segments having a second audio duration;

the second audio being substantially synchronized with the media stream;

the processor comparing the first audio duration with the second audio duration, where the processor compares each segment of the first audio stream with the corresponding segment of the second audio stream at run-time, and the processor adjusts the duration of the media stream based on the comparison.

62. A system as in claim 61 wherein the processor performs the comparison at regular intervals.

63. A system as in claim 61 wherein the processor adjusts the duration of the media stream to ensure that the media stream is substantially synchronized with the second audio stream.

64. A system for synchronizing media content comprising:

a media stream having a plurality of media segments, each media segment having a respective media duration;

a first audio having a plurality of first audio segments, each of the first audio segments having a respective first audio duration;

a second audio having a plurality of second audio segments, each of the second audio segments having a second audio duration;

the second audio being substantially synchronized with the media stream; and

the processor that automatically synchronizes the media stream to whichever audio is selected by adjusting the media duration of each segment, at run-time, to reflect the duration of the selected audio.