Synchronization aspects of interactive multimedia presentation management

• US 7,721,308 B2
• Filed: 02/16/2006
• Issued: 05/18/2010
• Est. Priority Date: 07/01/2005
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Finger, James C., Colle, Olivier, Yovin, John Andre
• Primary Examiner(s): Hillery; Nathan
• Application Number:
  US11/355,609
• Publication Number:
  US 20070006080A1
• Time in Patent Office: 1,552 Days
• Field of Search: 715/719, 715/202, 715/255, 725/37, 725/87
• US Class Current: 725/37
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/42646 for reading from or writing on a non-volatile solid state storage medium, e.g. DVD, CD-ROM
• CPC Class Codes: H04N 21/43074 of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/443 OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
• CPC Class Codes: H04N 21/47205 for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
• CPC Class Codes: H04N 21/8543 using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80
• CPC Class Codes: H04N 5/85 on discs or drums

Abstract

Playing an interactive multimedia presentation involves pre-rendering a media object at certain times based on a frame rate of the interactive content, a frame rate of the video content (which may include video, audio, data, or any combination thereof), and a play speed of the presentation. Certain actions taken include calculating a current elapsed play time representing an amount of the presentation'"'"'s play duration that has passed. Based on the calculated time, a current interactive content presentation time is ascertained from an interactive content timeline. The interactive content timeline represents times at which the media object is presentable. A subsequent interactive content presentation time is selected from the interactive content timeline. The presentation state of the media object is predicted for the subsequent interactive content presentation time. The media object is pre-rendered within a time offset period before the subsequent interactive content presentation time occurs.

Claims

1. A method for playing an interactive multimedia presentation, the interactive multimedia presentation having a play duration, a play speed, a video content component and an interactive content component, the video content component arranged into a plurality of video frames, the interactive content component comprising an interactive media object having a presentation state, the method comprising the following steps executed by at least a processor:

ascertaining a video frame rate representing a periodic time interval within which a video frame is presentable;

ascertaining an interactive content frame rate representing a periodic time interval within which the interactive media object having a presentation state is presentable;

based on the interactive content frame rate, ascertaining at least a portion of an interactive content timeline having interactive content presentation times representing times within the play duration at which the interactive media object is presentable;

based on the video frame rate and on the play speed, calculating a current elapsed play time representing an amount of time of the play duration that has passed;

based on the current elapsed play time, ascertaining a current interactive content presentation time from the interactive content timeline;

selecting a subsequent interactive content presentation time from the interactive content timeline, the subsequent interactive content presentation time occurring at a different time than the current interactive content presentation time;

predicting the presentation state of the interactive media object at the subsequent interactive content presentation time;

based on the predicted presentation state, pre-rendering the media object at a pre-rendering time having a predetermined time offset from the subsequent interactive content presentation time; and

arranging for presentation of the pre-rendered interactive media object at the subsequent interactive content presentation time.

2. The method according to claim 1, wherein the subsequent interactive content presentation time is selected by adding an amount based on an inverse of the interactive content frame rate to the current interactive content presentation time.

3. The method according to claim 1, wherein the predetermined time offset comprises an inverse of the interactive content frame rate.

4. The method according to claim 1, wherein the step of selecting a subsequent interactive content presentation time further comprises:

calculating a predicted elapsed time, the predicted elapsed time representing an amount of time of the play duration, in addition to the current elapsed play time, that has passed based on the play speed; and

selecting the subsequent interactive content presentation time based on the predicted elapsed time.

5. The method according to claim 4, further comprising:

based on the video frame rate, ascertaining at least a portion of a video timeline having frame number presentation times representing times within the play duration at which individual video frames having individual frame numbers are presentable.

6. The method according to claim 5, further comprising:

ascertaining a predicted frame number presentation time on the video timeline, the predicted frame number presentation time representing a frame number presentation time associated with the predicted elapsed time; and

selecting the subsequent interactive content presentation time based on the predicted frame number presentation time.

7. The method according to claim 6, wherein the step of ascertaining a predicted frame number presentation time on the video timeline comprises:

estimating a plurality of predicted frame number presentation times on the video timeline, and determining a pattern within the plurality of predicted frame number presentation times, the pattern based on the play speed, and

wherein the step of selecting the subsequent interactive content presentation time comprises selecting the subsequent interactive content presentation time based on the pattern.

8. The method according to claim 4, wherein the predicted elapsed time is calculated by adding to the current elapsed play time a multiplier value, the multiplier value calculated by multiplying a play speed factor by a frame rate factor, the play speed factor calculated by dividing a value representing the play speed by the video frame rate, and the frame rate factor calculated by dividing the video frame rate by the interactive content frame rate.

9. The method according to claim 8, wherein the value representing the play speed comprises one of a positive or a negative value.

10. The method according to claim 1, wherein the interactive content frame rate is the same as the video frame rate.

11. The method according to claim 1, wherein the interactive media object comprises one or more of an image, an audio sample, or text.

12. The method according to claim 1, wherein the video content component comprises samples selected from the group consisting of video, audio, and data, and wherein the step of ascertaining a video frame rate representing a periodic time interval within which one video frame is presentable comprises ascertaining a periodic time interval in which a group of samples of video, audio, or data is presentable.

13. The method according to claim 1, wherein the presentation state of the interactive media object comprises one of on or off.

14. A computer-readable storage medium encoded with computer-executable instructions that, when executed, cause a computing device to play an interactive multimedia presentation by:

ascertaining a video frame rate representing a periodic time interval within which a video frame is presentable;

ascertaining an interactive content frame rate representing a periodic time interval within which a media object having a presentation state is presentable, wherein the presentation state determines whether or how the media object is presentable;

based on the interactive content frame rate, ascertaining at least a portion of an interactive content timeline having interactive content presentation times representing times within a play duration at which the media object is presentable;

based on the video frame rate and on a play speed, calculating a current elapsed play time representing an amount of time of the play duration that has passed;

based on the current elapsed play time, ascertaining a current interactive content presentation time from the interactive content timeline;

selecting a subsequent interactive content presentation time from the interactive content timeline, the subsequent interactive content presentation time occurring at a different time than the current interactive content presentation time;

predicting the presentation state of the media object at the subsequent interactive content presentation time;

based on the predicted presentation state, pre-rendering the media object at a pre-rendering time having a predetermined time offset from the subsequent interactive content presentation time; and

arranging for presentation of the pre-rendered media object at the subsequent interactive content presentation time.

15. A system for playing an interactive multimedia presentation, comprising:

at least a processor for executing the interactive multimedia presentation, the interactive multimedia presentation having a play duration, a play speed, and a computer storage medium having stored thereon at least a video content component and an interactive content component, wherein the video content component arranged into a plurality of video frames, the interactive content component comprising a media object having a presentation state, the system further comprising;

an interactive content manager configured to arrange the media object for rendering, based on the presentation state, at an interactive content rendering rate;

a video content manager configured to arrange the individual video frames for rendering at a video content rendering rate;

a time reference calculator operable to measure a current elapsed play time representing an amount of time of the play duration that has passed, based on the video content rendering rate and on the play speed; and

a presentation manager configured for communication with the interactive content manager and the video content manager and responsive to receive the current elapsed play time from the time reference calculator, the presentation manager operable to

based on the interactive content rendering rate, ascertain at least a portion of an interactive content timeline having interactive content presentation times representing times within the play duration at which the media object is presentable,

based on the current elapsed play time, ascertain a current interactive content presentation time from the interactive content timeline, and

select a subsequent interactive content presentation time from the interactive content timeline, the subsequent interactive content presentation time occurring at a different time than the current interactive content presentation time,

based on a predicted presentation state of the media object at the subsequent interactive content presentation time, the interactive content manager operable to pre-render the media object at a pre-rendering time having a predetermined time offset from the subsequent interactive content presentation time, and operable to arrange for presentation of the pre-rendered media object at the subsequent interactive content presentation time.

16. The system according to claim 15, wherein the presentation manager is further operable to calculate a predicted elapsed time, the predicted elapsed time representing an amount of time of the play duration, in addition to the current elapsed play time, that has passed based on the play speed, and operable to select the subsequent interactive content presentation time based on the predicted elapsed time.

17. The system according to claim 15, wherein the system comprises an operating system.

18. The system according to claim 17, wherein the operating system is associated with an optical disc player.

19. The system according to claim 18, wherein the optical disc player complies with specifications for high definition video published by the DVD Forum.

20. The system according to claim 17, wherein the operating system is associated with an electronic device.