Timing aspects of media content rendering

• US 7,861,150 B2
• Filed: 11/07/2006
• Issued: 12/28/2010
• Est. Priority Date: 11/07/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Colle, Oliver, Yovin, John Andre, Guinan, Cathleen M., Finger, James C.
• Primary Examiner(s): Hutton; Doug
• Assistant Examiner(s): Hillery; Nathan
• Application Number:
  US11/594,352
• Publication Number:
  US 20080109727A1
• Time in Patent Office: 1,512 Days
• Field of Search: 715/205, 715/719, 715/723
• US Class Current: 715/203
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: H04N 21/42646 for reading from or writing on a non-volatile solid state storage medium, e.g. DVD, CD-ROM
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4431 characterized by the use of Application Program Interface [API] libraries
• CPC Class Codes: H04N 21/8545 for generating interactive applications
• CPC Class Codes: H04N 5/85 on discs or drums
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

Timing for execution of certain user inputs and application instructions occurring during play of an interactive multimedia presentation is discussed. A current state is defined by a state of the presentation system at the time a current media sample is being played to a user. A predicted state is defined by a state of the presentation system one or more future play times. Examples of current and predicted states include media retrieval states and media presentation states. An instruction or user input that is based on the current state is identified, and the predicted state is used to determine an effect of the instruction or input. The effect may then be executed at a predetermined time, such as after the next playable media sample is played to the user.

Claims

1. A computer-readable storage medium encoded with computer-executable instructions which, when executed by a processor, perform a method for playing an interactive multimedia presentation using a presentation system, the interactive multimedia presentation having a play duration, a media content component arranged into a plurality of media samples receivable from a media source, and having an interactive content component, an application having computer-executable instructions for playing the interactive content component, the method comprising:

ascertaining at least a portion of a media timeline having media sample presentation times representing times within the play duration at which at least one media sample is playable to a user;

with reference to the media timeline, identifying a current elapsed play time representing a time within the play duration when a current media sample from a current media source is playable to a user;

ascertaining a current state associated with a state of the presentation system at the current elapsed play time;

with reference to the media timeline, identifying a predicted elapsed play time within the play duration at which a next playable media sample from a next media source is playable to the user;

ascertaining a predicted state associated with a state of the presentation system at the predicted elapsed play time;

identifying an instruction or a user input that is scheduled for execution based on the current state, the instruction able to cause loss of frame-level synchronization between the media content component and the interactive content component; and

prior to the predicted elapsed play time arising, arranging for execution of the instruction or the user input based on the predicted state instead of the current state, to maintain frame-level synchronization between the media content component and the interactive content component.

2. The computer-readable storage medium according to claim 1, wherein the next playable media sample is not the next consecutive media sample.

3. The computer-readable storage medium according to claim 1, further comprising:

identifying a user input occurring after the current elapsed play time, the predicted state based on the user input.

4. The computer-readable storage medium according to claim 3, wherein the user input is selected form the group consisting of:

a play speed input; and

a selection of an interactive object associated with the interactive content component.

5. The computer-readable storage medium according to claim 1, wherein the current state and the predicted state are selected from the group consisting of:

the current media sample;

the current media source;

a next playable media sample; and

a next media source.

6. The computer-readable storage medium according to claim 1, wherein the predicted state comprises a retrieval state associated with a future media source.

7. The computer-readable storage medium according to claim 1, wherein execution of the instruction modifies the current state or is conditioned upon the current state.

8. The computer-readable storage medium according to claim 1, wherein the step of arranging for execution of the instruction occurs after the current elapsed play time arises.

9. The computer-readable storage medium according to claim 8, wherein the current media sample is being rendered to a user, and wherein the next playable media sample is being prepared for rendering to the user.

10. The computer-readable storage medium according to claim 1, wherein the instruction is associated with an application programming interface (“

API”

) used by an author of the application.

11. The computer-readable storage medium according to claim 10, wherein the step of arranging for execution of the instruction is performed by an implementation of the API.

12. The computer-readable storage medium according to claim 1, wherein the instruction comprises an imperative instruction.

13. An apparatus for playing an interactive multimedia presentation, the interactive multimedia presentation having a play duration, a media content component arranged into a plurality of media samples receivable from a media source, and having an interactive content component, an application having computer-executable instructions for playing the interactive content component, the apparatus comprising:

a computer-readable storage medium; and

a processor responsive to the computer-readable storage medium and to a computer program stored in the computer-readable storage medium, the computer program, when loaded into the processor and executed, operable to

ascertain at least a portion of a media timeline having media sample presentation times representing times within the play duration at which at least one media sample is playable to a user,

with reference to the media timeline, identify a current elapsed play time representing a time within the play duration when a current media sample from a current media source is playable to a user,

with reference to the media timeline, identify one or more future play times representing times within the play duration at which one or more future playable media samples from one or more future media sources are playable to a user,

identify an application instruction scheduled for execution after the current elapsed play time arises but before the one or more future play times arise, execution of the instruction able to cause loss of frame-level synchronization between the media content component and the interactive content component and

prior to the one or more future play times arising, query a data structure about a first state value associated with a current state, the data structure comprising

a first data portion having information identifying the first state value, the first state value associated with the current state, the current state associated with the current media sample or the current media source, and

a second data portion having information identifying a second state value, the second state value associated with a predicted state, the predicted state associated with one or more of the future media samples or the future media sources,

the data structure returning the second state value in response to queries about the first state value;

based on the query, prior to the one or more future play times arising, execute the application instruction based on the second state value, to maintain frame-level synchronization between the media content component and the interactive content component.