Synchronization aspects of interactive multimedia presentation management

• US 8,799,757 B2
• Filed: 02/15/2006
• Issued: 08/05/2014
• Est. Priority Date: 07/01/2005
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Finger, James C., Colle, Olivier, Hughes, Robert K. Jr., Hayes, Sean, Tan, See-Mong, Jewsbury, Andrew William
• Primary Examiner(s): Paula, Cesar
• Assistant Examiner(s): Faber, David
• Application Number:
  US11/355,209
• Publication Number:
  US 20070006063A1
• Time in Patent Office: 3,093 Days
• Field of Search: 715/500.1, 715/255, 715/255.I, 715200-203
• US Class Current: 715/203
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/44 Browsing; Visualisation therefor
• CPC Class Codes: G11B 2220/2541 Blu-ray discs; Blue laser DVR discs
• CPC Class Codes: G11B 2220/2579 HD-DVDs [high definition DVDs]; AODs [advanced optical discs]
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/322 used signal is digitally coded
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56
• CPC Class Codes: H04N 5/85 on discs or drums

Abstract

An interactive multimedia presentation has a predetermined presentation play duration, a video content component (which may include video, audio, data, or any combination thereof), and an interactive content component. Methods and systems for playing the presentation include specifying a time when a particular media object is renderable. The time is specifiable using an application instruction that references either a first timing signal having a rate that is based on the play speed of the presentation, or a second timing signal having a continuous rate. One example of an application instruction usable as described above is an element associated with an XML data structure. An attribute of the element, such as the clock attribute, may be used to provide time values for the element that reference either timing signal. The attribute is used to maintain frame-accurate synchronization between the video and the interactive content components of the presentation.

Claims

1. An apparatus for playing an interactive multimedia presentation including audio/video content and interactive content, comprising:

one or more real processors; and

memory storing computer-readable data including instructions executable by the one or more real processors to implement

an audio/video content manager configured for producing frames of audio/video content that are renderable as part of the interactive multimedia presentation, the audio/video content being renderable at a play speed that is selectable according to user input, the play speed being variable between intervals of normal playback of the audio/video content and trick play of the audio/video content,

an interactive content manager configured for producing frames of interactive content that are renderable as part of the interactive multimedia presentation, the interactive content including media objects being generated by one or more applications,

a timing signal management block configured for generating a play speed-based timing signal and a continuous timing signal,

the play speed-based timing signal having a variable rate based on the play speed and being utilized for determining an audio/video frame rate at which the frames of audio/video content are produced and rendered,

the continuous timing signal having a constant rate which is invariant with respect to the play speed, and being utilized for determining an interactive content frame rate at which the frames of interactive content are produced and rendered, and

a presentation manager in communication with the audio/video content manager, interactive content manager, and timing signal management block, and being configured for determining a presentation timeline identifying times for which clips of audio/video content are presentable to one or more users of the apparatus, the times being identified using a timeline that is based on the play speed-based timing signal, in which the presentation timeline includes one of a title timeline or a media timeline, each of the title and media timelines being paused or resumed as the video content is paused or resumed, and time T on each of the title and media timelines elapsing at N×

T as the play speed of the video content changes by a factor of N, the title timeline providing a total elapsed play time for all clips of audio/video content and the media timeline providing a total elapsed play time for a given clip of audio/video content.

2. The apparatus of claim 1 in which the presentation manager is configured for determining a presentation timeline identifying times for which media objects generated by the one or more applications are presentable to one or more users of the apparatus, the times being identified using a timeline that is based on the continuous timing signal.

4. The apparatus of claim 1 further comprising

a mixer/renderer configured for rendering frames of interactive into a graphics plane that is overlaid on a video plane into which video content frames are rendered so that the media objects are rendered synchronously with the audio/video content.

3. The apparatus of claimed 2 in which the psresentation timeline includes one of an application timeline or page timeline, each of the application and page timelines continuing irrespective of pausing and resuming of the video content, trick play, or changes in the video content play speed, the application timeline providing a play duration for an application and the page timeline providing a play duration for a page of the application.

5. A method for playing an interactive multimedia presentation including audio/video content and interactive content, the method comprising:

producing frames of audio/video content that are renderable as part of the interactive multimedia presentation, the audio/video content being renderable at a play speed that is selectable according to user input, the play speed being variable between intervals of normal playback of the audio/video content and trick play of the audio/video content;

producing frames of interactive content that are renderable as part of the interactive multimedia presentation, the interactive content including media objects being generated by one or more applications;

generating a play speed-based timing signal and a continuous timing signal, the play speed-based timing signal having a variable rate based on the play speed and being utilized for determining an audio/video frame rate at which the frames of audio/video content are produced and rendered, the continuous timing signal having a constant rate which is invariant with respect to the play speed, and being utilized for determining an interactive content frame rate at which the frames of interactive content are produced and rendered; and

determining a presentation timeline identifying times for which clips of audio/video content are presentable to one or more users, the times being identified using a timeline that is based on the play speed-based timing signal, in which the presentation timeline includes one of a title timeline or a media timeline, each of the title and media timelines being paused or resumed as the video content is paused or resumed, and time T on each of the title and media timelines elapsing at N×

T as the play speed of the video content changes by a factor of N, the title timeline providing a total elapsed play time for all clips of audio/video content and the media timeline providing a total elapsed play time for a given clip of audio/video content.

6. The method of claim 5 further comprising determining a presentation timeline identifying times for which media objects generated by the one or more applications are presentable to one or more users, the times being identified using a timeline that is based on the continuous timing signal.

7. The method of claim 6 in which the presentation timeline includes one of an application timeline or page timeline, each of the application and page timelines continuing irrespective of pausing and resuming of the video content, trick play, or changes in the video content play speed, the application timeline providing a play duration for an application and the page timeline providing a play duration for a page of the application.

8. The method of claim 5 further comprising rendering frames of interactive content into a graphics plane that is overlaid on a video plane into which video content frames are rendered so that the media objects are rendered synchronously with the audio/video content.

9. A memory storing computer-readable data including instructions which, when executed by one or more real processors, implement:

an audio/video content manager configured for producing frames of audio/video content that are renderable as part of the interactive multimedia presentation, the audio/video content being renderable at a play speed that is selectable according to user input, the play speed being variable between intervals of normal playback of the audio/video content and trick play of the audio/video content,

an interactive content manager configured for producing frames of interactive content that are renderable as part of the interactive multimedia presentation, the interactive content including media objects being generated by one or more applications,

a timing signal management block configured for generating a play speed-based timing signal and a continuous timing signal,

the play speed-based timing signal having a variable rate based on the play speed and being utilized for determining an audio/video frame rate at which the frames of audio/video content are produced and rendered,

the continuous timing signal having a constant rate which is invariant with respect to the play speed, and being utilized for determining an interactive content frame rate at which the frames of interactive content are produced and rendered, and

a presentation manager in communication with the audio/video content manager, interactive content manager, and timing signal management block, and being configured for determining a presentation timeline identifying times for which clips of audio/video content are presentable to one or more users, the times being identified using a timeline that is based on the play speed-based timing signal, in which the presentation timeline includes one of a title timeline or a media timeline, each of the title and media timelines being paused or resumed as the video content is paused or resumed, and time T on each of the title and media timelines elapsing at N×

T as the play speed of the video content changes by a factor of N, the title timeline providing a total elapsed play time for all clips of audio/video content and the media timeline providing a total elapsed play time for a given clip of audio/video content.

10. The memory of claim 9 in which the presentation manager is configured for determining a presentation timeline identifying times for which media objects generated by the one or more applications are presentable to one or more users, the times being identified using a timeline that is based on the continuous timing signal.

11. The memory of claim 10 in which the presentation timeline includes one of an application timeline or page timeline, each of the application and page timelines continuing irrespective of pausing and resuming of the video content, trick play, or changes in the video content play speed, the application timeline providing a play duration for an application and the page timeline providing a play duration for a page of the application.

12. The memory of claim 9 further comprising

a mixer/renderer configured for rendering frames of interactive content into a graphics plane that is overlaid on a video plane into which video content frames are rendered so that the media objects are rendered synchronously with the audio/video content.