DATA MODEL AND PLAYER PLATFORM FOR RICH INTERACTIVE NARRATIVES

• US 20110119587A1
• Filed: 01/18/2011
• Published: 05/19/2011
• Est. Priority Date: 12/31/2008
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Stollnitz, Eric J., Joy, Joseph M., Datha, Narendranath
• Application Number:
  US13/008,324
• Publication Number:
  US 20110119587A1
• Time in Patent Office: Days
• US Class Current: 715/720
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/322 used signal is digitally coded
• CPC Class Codes: H04N 21/2665 Gathering content from different sources, e.g. Internet and satellite
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859

Abstract

A data model and player platform for playing rich interactive narratives (RINs) is presented. Together, they enable a very broad class of rich interactive applications in a device independent way that is also platform technology proof and can be extended to new kinds of interactive visualization technologies. The RIN data model includes a narrative having a prescribed sequence of scenes, where each scene is made up of one or more RIN segments. Each of the RIN segments includes one or more experience streams (or references thereto), and at least one screenplay. Each experience stream includes data that enables a user employing a RIN player to traverse a particular environment created by an arbitrary media type. In addition, each screenplay includes data to orchestrate when each experience stream starts and stops during the playing of the RIN and to specify how experience streams share display screen space or audio playback configuration.

Claims

1. A system for processing rich interactive narrative (RIN) data to provide a narrated traversal of arbitrary media types and user-explorable content of the media, said data stored on a computer-readable storage medium which is accessible during a play-time by a RIN player running on a user'"'"'s computing device, said system comprising:

RIN data which is input to said user computing device and stored on said computer-readable storage medium, wherein the RIN data comprises a narrative comprising a prescribed sequence of scenes, wherein each scene is comprised of one or more RIN segments, each of said RIN segments comprising one or more experience streams or references thereto, and at least one screenplay, wherein each experience stream comprises data that enables traversing a particular environment created by a one of said arbitrary media types whenever the RIN segment is played, and each screenplay comprises data to orchestrate when each experience stream starts and stops during the playing of the RIN data and specify how experience streams share display screen space, or audio playback configuration, or both; and

said RIN player which accesses and processes the RIN data to play a RIN to the user via an audio playback device, or video display device, or both, associated with the user'"'"'s computing device.

2. The system of claim 1, wherein a RIN segment comprises more than one screenplay, and wherein only one screenplay is employed by the RIN player at a time, and wherein each screenplay represents a different language-specific or culture-specific interpretation of the RIN segment from the other screenplays.

3. The system of claim 1, wherein said screenplay data further comprises at least one of:

instructions whereby only a portion of an experience stream is played rather than the whole stream;

or

instructions enabling user interactivity with an experience stream;

or

instructions disabling previously enabled user interactivity with an experience stream.

4. The system of claim 1, wherein each screenplay comprises layout constraints for each experience stream in the same RIN segment as the screenplay, and wherein the layout constraints for an experience stream dictate how the visual or audio elements, or both, associated with that experience stream share display screen space or audio playback configuration with the visual or audio elements associated with the other experience steams in the RIN segment, as a function of time.

5. The system of claim 4, wherein said experience stream data is used to generate a virtual viewport through which a user sees, or hears, or both, the environment associated with an experience stream as the user traverses the environment.

6. The system of claim 5, wherein the layout constraints for an experience stream dictate how the visual elements associated with that experience stream share display screen space with the visual elements associated the other experience steams in the RIN segment by indicating the experience stream'"'"'s virtual viewport'"'"'s z-order, 2D layout and size.

7. The system of claim 6, wherein the layout constraints for an experience stream further dictate how the visual elements associated with that experience stream share display screen space with the visual elements associated the other experience steams in the RIN segment by indicating the experience stream'"'"'s virtual viewport'"'"'s opacity.

8. The system of claim 5, wherein the layout constraints for an experience stream dictate how the audio elements associated with that experience stream share the audio playback configuration with the audio elements associated the other experience steams in the RIN segment by specifying the relative audio mix levels of the experience streams.

9. The system of claim 1, wherein each of said RIN segments further comprises a resource table comprising reference metadata that enables references to external media in an experience stream or shared across experience streams to be obtained.

10. The system of claim 1, wherein said experience stream data comprises data bindings and a trajectory, wherein the data bindings refer to static or dynamically queried data that defines and populates the environment associated with the experience stream, and which comprises environment data, artifacts and highlighted regions, and wherein the trajectory defines a traversal of the environment associated with the experience stream or the evolution of the environment itself, and which comprises a sequence of keyframes, transitions and markers.

11. The system of claim 10, wherein said artifacts comprise objects that appear or are heard in the environment generated from the environment data as the experience stream associated with the environment data and artifact is played, and wherein said highlighted regions comprise regions that are visibly highlighted in the environment generated from the environment data as the experience stream associated with the environment data and highlighted region is played.

12. The system of claim 11, wherein said artifacts and highlighted regions are associated with metadata used to annotate content in the environment or describe the artifact or highlighted region, and wherein said artifacts and highlighted regions appear, disappear and change over time as the experience stream associated with the artifacts and highlighted regions is played.

13. The system of claim 10, wherein said experience stream data is used to generate a virtual viewport through which a user sees, or hears, or both, the environment associated with an experience stream as the user traverses the environment, and wherein each keyframe in said sequence of keyframes captures the information state of the experience stream at a particular point of time, and provides an environment-to-viewport mapping at that point in time.

15. The system of claim 10, wherein said markers mark a particular point in time in an experience stream'"'"'s trajectory, and are used for at least one of indexing content by searching over indexed markers, or semantic annotation by associating metadata with particular times in the trajectory, or generalized synchronization, or triggering of events, or triggering of a decision point where user input is solicited, or acting as logical anchors that refer to external references that are introduced at the marked time in the trajectory.

14. The system of Clam 10, wherein said transitions define how inter-keyframe interpolations are done, and are broadly classified into smooth or discontinuous categories.

16. A system for playing rich interactive narrative (RIN) data to provide a narrated traversal of arbitrary media types and user-explorable content of the media, comprising:

a computing device comprising audio playback equipment, a display device and a user interface input device;

a computer program comprising program modules executed by the computing device, said program modules comprising,

a presentation platform module which provides a user interface that allows a user to view visual components and hear audio components of the narrated traversal and user-explorable media content via the display device and audio playback equipment, and to input data and commands via the user interface input device, and through which the user accesses the RIN data in the form of a RIN file,

an orchestrator module which accesses a screenplay from the RIN file, and identifies and loads a pluggable screenplay interpreter module that understands how to interpret the particular format of the screenplay, and which thereafter follows instructions received from the screenplay interpreter module,

said pluggable screenplay interpreter module which identifies one or more pluggable experience stream provider modules which are capable of playing experience streams, and instructs the orchestrator module to access and load the identified experience stream provider modules, and which further employs orchestration information found in said screenplay to determine a layout for each experience stream defining how the visual or audio components, or both, associated with that experience stream are to be displayed and heard on the display device and a timing of when each of the experience streams starts and ends playing, and instructs the orchestrator module, on an on-going basis, to cause the experience stream provider modules to commence and stop playing each instance of the experience streams at the determined times, and which still further instructs the orchestrator module, on an on-going basis, to cause the experience stream provider modules to render the visual or audio components, or both, associated with each experience stream in accordance with the determined layout for that experience stream, and

one or more of said pluggable experience stream provider modules which create instances of experience streams using the experience stream data and a resource table found in the RIN file in response to instructions from the orchestrator module, wherein the resource table comprises metadata used to access external media needed along with the experience stream data to create the instances of the experience streams.

17. The system of claim 16, wherein whenever a user employs said user interface input device to input a pause command, or the playing of the RIN data is paused to allow for the download and buffering of additional RIN data or associated media, the screenplay interpreter module instructs the orchestrator module to cause the experience stream provider modules associated with the currently playing experience streams to stop playing the streams, and whenever a user employs said user interface input device to input a restart command, the screenplay interpreter module instructs the orchestrator module to cause the experience stream provider modules to restart the previously paused experience streams.

18. The system of claim 16, wherein a current state of currently playing experience streams is reported and stored via an eventing and state sharing mechanism, and wherein whenever an experience stream comprises a triggering event that involves a cessation of the playing of currently playing experience streams and starting the playing of one or more of the same streams at a different point, or starting the playing of different streams, or both, the screenplay interpreter module instructs the orchestrator module to cause the experience stream provider modules to accomplish the stopping and starting of experience streams in accordance with said triggering event based on the current states of the experience stream currently stored via the eventing and state sharing mechanism.

19. The system of claim 16, wherein the experience stream provider modules employ a preloading logic to preload external media in the chronological order that it is needed during the playing of the associated experience stream and in a manner giving priority to media to be used in the immediate future.

20. A computer-implemented process for playing a RIN segment in a rich interactive narrative (RIN) comprising multiple sequential RIN segments using a RIN player comprising a presentation platform, an orchestrator, a pluggable screenplay interpreter and one or more pluggable experience stream providers, comprising:

using a computer to perform the following process actions;

using the orchestrator to load a screenplay associated with the segment being played;

using the orchestrator to identify, request via the presentation platform, input via the presentation platform and plug-in a screenplay interpreter applicable to the particular format of the screenplay;

using the screenplay interpreter to identify the experience stream providers needed to play experience streams called out in the RIN segment being played;

using the screenplay interpreter to instruct the orchestrator to request via the presentation platform, input via the presentation platform and plug-in each of the identified experience stream providers;

using the orchestrator to request, input and plug-in each of the identified experience stream providers;

using the screenplay interpreter to further instruct the orchestrator to have each experience stream provider create an instance of an experience stream associated with that provider using experience stream data found in the RIN segment being played;

using the orchestrator to cause each experience stream provider to create an instance of the experience stream associated with that provider;

using the screenplay interpreter to determine a layout and timing of each of the experience streams using orchestration information found in the screenplay, monitoring events that affect the layout and timing of each of the experience streams, and instruct the orchestrator on an ongoing basis as to when to commence and stop playing each of the experience streams based on the timing data associated with that experience stream, and changes thereto;

using the screenplay interpreter to instruct the orchestrator as to the layout of each experience stream that is running based on the layout data associated with that experience stream, and changes thereto;

using, for each experience stream, the orchestrator to cause the experience stream provider associated with the experience stream to start and stop playing that stream at the times specified by the screenplay interpreter, and to cause the presentation platform to layout each experience stream in a presentation platform window while it is playing in accordance with the layout instructions for that stream.