System and method for synchronization of selectably presentable media streams

• US 10,418,066 B2
• Filed: 12/30/2015
• Issued: 09/17/2019
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: JBF Interlude 2009 Ltd.
• Original Assignee: JBF Interlude 2009 Ltd.
• Inventors: Bloch, Yoni, Zubalsky, Tal, Feldman, Barak
• Primary Examiner(s): Hunter, Mishawn N.
• Application Number:
  US14/984,821
• Publication Number:
  US 20160217829A1
• Time in Patent Office: 1,357 Days
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/47217 for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks specific graphical features in visual interfaces H04N21/4312
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content
• CPC Class Codes: H04N 5/9305 involving the mixing of the reproduced video signal with a non-recorded signal, e.g. a text signal

Abstract

A system for synchronizing audio and video of selectably presentable multimedia content includes a memory for storing a plurality of selectably presentable multimedia content segments. Each content segment defines a portion of one or more content paths and includes a decision period during which a user may select a subsequent content segment as the content segment is playing. An assembly engine seamlessly assembles a subset of the content segments into one of the content paths, ultimately forming a multimedia presentation. A configuration manager determines an audio file and a video file to be played based on a content segment that is selected to be played immediately following the currently playing content segment. An audio engine processes the audio file for playback, and a video engine synchronizes playback of the video file with the playback of the audio file.

Claims

1. A system comprising:

a memory for storing a plurality of multimedia content segments, each content segment defining a portion of one or more content paths, wherein a particular content segment comprises a decision period during which a subsequent content segment can be selected for playback as the particular content segment is playing;

an assembly engine for seamlessly assembling a subset of the content segments into a multimedia presentation comprising one of the content paths;

a configuration manager for determining an audio file and a video file to be played based on a selected subsequent content segment;

an audio engine for;

dividing the audio file into a plurality of audio samples, each audio sample comprising a timestamp indicating a time at which the audio sample should be played; and

upon causing one of the audio samples to be played, updating a current playback time based on the timestamp of the played audio sample, wherein the current playback time comprises a time that is maintained separately from timestamps of the audio samples and identifies a most recently played audio sample; and

a video engine for synchronizing playback of the video file with playback of the audio file based on the current playback time.

2. The system of claim 1, wherein synchronizing playback of the video file with playback of the audio file comprises:

dividing the video file into a plurality of video frames, each video frame comprising a timestamp indicating a time at which the video frame should be played;

comparing the timestamp of at least one of the video frames with the current playback time; and

determining when to play the at least one of the video frames based on the comparing.

3. The system of claim 2, wherein comparing the timestamp comprises determining that the timestamp of the at least one of the video frames is substantially close to the current playback time, and wherein determining when to play the at least one of the video frames comprises causing the at least one of the video frames to be immediately played.

4. The system of claim 2, wherein comparing the timestamp comprises determining that the timestamp of the at least one of the video frames is earlier than the current playback time, and wherein determining when to play the at least one of the video frames comprises discarding the at least one of the video frames.

5. The system of claim 4, wherein the video engine is further for repeating the comparing and discarding functions for consecutive video frames until reaching a video frame having a timestamp that is substantially close to the current playback time.

6. The system of claim 5, wherein the video engine is further for determining that a maximum number of consecutive video frames has been discarded, and causing a next video frame to be played irrespective of a timestamp associated with the next video frame.

7. The system of claim 2, wherein comparing the timestamp comprises determining that the timestamp of the at least one of the video frames is later than the current playback time, and wherein determining when to play the at least one of the video frames comprises delaying playback of the at least one of the video frames.

8. The system of claim 1, wherein the memory is further for storing a plurality of audio files, a plurality of video files, and a configuration file;

and wherein determining the audio file and the video file to be played comprises;

selecting the audio file from the plurality of audio files based on the configuration file and the selected subsequent content segment; and

selecting the video file from the plurality of video files based on the configuration file and the selected subsequent content segment.

9. The system of claim 1, further comprising a graphical user interface manager for providing, during the decision period as the particular content segment is playing, visual representations of two or more alternative content segments from which a subsequent content segment can be selected.

10. The system of claim 9, wherein the graphical user interface manager is further for providing, upon reaching the decision period, a visual representation of a time interval representing an amount of time remaining to select one of the alternative content segments.

11. A method comprising:

storing, in a memory, a plurality of multimedia content segments, each content segment defining a portion of one or more content paths, wherein a particular content segment comprises a decision period during which a subsequent content segment can be selected for playback as the particular content segment is playing;

determining, by a configuration manager, an audio file and a video file to be played based on a selected subsequent content segment;

dividing, by an audio engine, the audio file into a plurality of audio samples, each audio sample comprising a timestamp indicating a time at which the audio sample should be played;

seamlessly joining, by an assembly engine, the selected subsequent content segment to a currently playing content segment;

upon causing one of the audio samples to be played, updating, by the audio engine, a current playback time based on the timestamp of the played audio sample, wherein the current playback time comprises a time that is maintained separately from timestamps of the audio samples and identifies a most recently played audio sample; and

synchronizing, by a video engine, playback of the video file with the playback of the audio file based on the current playback time.

12. The method of claim 11, wherein synchronizing playback of the video file with the playback of the audio file comprises:

dividing the video file into a plurality of video frames, each video frame comprising a timestamp indicating a time at which the video frame should be played;

comparing the timestamp of at least one of the video frames with the current playback time; and

determining when to play the at least one of the video frames based on comparing.

13. The method of claim 12, wherein comparing the timestamp comprises determining that the timestamp of the at least one of the video frames is substantially close to the current playback time, and wherein determining when to play the at least one of the video frames comprises causing the at least one of the video frames to be immediately played.

14. The method of claim 12, wherein comparing the timestamp comprises determining that the timestamp of the at least one of the video frames is earlier than the current playback time, and wherein determining when to play the at least one of the video frames comprises discarding the at least one of the video frames.

15. The method of claim 14, further comprising repeating, by the video engine, the comparing and discarding functions for consecutive video frames until reaching a video frame having a timestamp that is substantially close to the current playback time.

16. The method of claim 15, further comprising determining, by the video engine, that a maximum number of consecutive video frames has been discarded, and causing a next video frame to be played irrespective of a timestamp associated with the next video frame.

17. The method of claim 12, wherein comparing the timestamp comprises determining that the timestamp of the at least one of the video frames is later than the current playback time, and wherein determining when to play the at least one of the video frames comprises delaying playback of the at least one of the video frames.

18. The method of claim 11, further comprising storing, in the memory, a plurality of audio files, a plurality of video files, and a configuration file;

and wherein determining the audio file and the video file to be played comprises;

selecting the audio file from the plurality of audio files based on the configuration file and the selected subsequent content segment; and

selecting the video file from the plurality of video files based on the configuration file and the selected subsequent content segment.

19. The method of claim 11, further comprising providing, by a graphical user interface manager, during the decision period as the particular content segment is playing, visual representations of two or more alternative content segments from which a subsequent content segment can be selected.

20. The method of claim 19, further comprising providing, by the graphical user interface manager, upon reaching the decision period, a visual representation of a time interval representing an amount of time remaining to select one of the alternative content segments.