Hyperstories: organizing multimedia episodes in temporal and spatial displays

• US 5,742,283 A
• Filed: 04/15/1997
• Issued: 04/21/1998
• Est. Priority Date: 09/27/1993
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kim, Michelle Yoonkyung Lee
• Primary Examiner(s): Burwell, Joseph R.
• Application Number:
  US08/843,312
• Time in Patent Office: 371 Days
• Field of Search: 395/806, 395/807, 395/762, 395/776-778, 395/779, 395/780, 395/802
• US Class Current: 715/201
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/45 Clustering; Classification
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence

Abstract

The design, creation, organization, and playing of multimedia stories is accomplished by first defining a story with episodes scheduled on a temporal layout. The organization of the story episodes is then separated into two parts: (1) a temporal organization of story episodes using temporal cliques, and (2) a spatial organization which spatially organizes the members of each temporal clique independently of the members of other temporal cliques. A complicated multimedia problem that exists in time and space is reduced to a number of smaller problems that exist only in space. Stories that are organized according this way can be related by a directed graph into a hyperstory. Using the directed graph, different stories in the hyperstory can be played by satisfying defined asynchronous conditions. The conditions are used in the graph to define the relationships among the stories in the graph.

Claims

1. A method of organizing multimedia segments, the multimedia segments having one or more sensory qualities when played on a multimedia computer system, the multimedia segments being stored on a memory of the computer system, and the multimedia segments being organized to play on the computer system, the method being executed by the computer system and comprising the steps of:

selecting one or more multimedia segments from a plurality of segments, the selected segments each having a temporal component including a start, an end, and a duration, the selected segments being episodes of a story;

scheduling the story episodes in a schedule sequence that defines the story;

grouping the story episodes into one or more temporal cliques, each of one or more first episodes in each temporal clique overlapping one or more of the other episodes in the temporal clique by having any one of the start and end of the first episode scheduled at a time in the schedule sequence that is the same as any one of the start, end, and duration of at least one, but not all, of the other episodes in the temporal clique, at least one of the temporal components of at least one of the first episodes being different than the respective temporal component of one or more of the other episodes;

spatially organizing the episodes in each temporal clique independently of the episodes in other temporal cliques, the spatial organizing placing any of the first episodes in the temporal clique in a spatial organization so that the first episode does not clash with any of the other episodes in the temporal clique when the episodes in the temporal clique are played, a clash occurring when a first multimedia segment plays at the same time as one or more other multimedia segments and the first multimedia segment interferes with the one or more of the other multimedia segments; and

playing the episodes of the story in the schedule sequence with the spatial organization of the temporal cliques.

2. A method of organizing multimedia story episodes, as in claim 1, where the schedule sequence is a temporal layout.

3. A method of organizing multimedia story episodes, as in claim 2, where the temporal layout schedules the episodes in the schedule sequence according to one or more relationships among the episodes.

4. A method of organizing multimedia story episodes, as in claim 3, where one or more temporal cliques is spatially organized by initially placing the episodes in the temporal clique in an initial spatial organization and then negotiating one or more space requirements among the episodes to obtain a final spatial organization.

5. A method of organizing multimedia story episodes, as in claim 3, where the spatial organization is done by a user using a graphical interface.

6. A method of organizing multimedia story episodes, as in claim 3, where the spatial organization of one or more temporal cliques is played by being displayed as a visual display of the computer system.

7. A method of organizing multimedia story episodes, as in claim 6, where the visual display is a computer screen.

8. A method of organizing and playing two or more multimedia stories as a hyperstory, the hyperstory being two or more multimedia stories organized to play on a multimedia computer system, the method being executed by the multimedia computer system and comprising the steps of:

creating two or more multimedia stories by selecting one or more multimedia segments, the multimedia segments having one or more sensory qualities when played on the multimedia computer system, the multimedia segments being stored on a memory of the computer system, and each of the selected multimedia segments being a story episode of a set of story episodes comprising of one of the stories;

scheduling the story episodes of each of the stories according to a temporal layout that defines the story;

grouping the story episodes of each story into one or more temporal cliques, each of one or more first story episodes in each temporal clique of the story overlapping other episodes in their respective temporal clique by having a part of a first duration of the first story episode scheduled at a time in the temporal layout that is the same as a part of the duration of at least one, but not all, of the other episodes in the temporal clique, at least one of the first durations not starting at the same time as the duration of at least one or more of the other episodes in the temporal clique;

spatially organizing the episodes in each temporal clique independently of the episodes in the other temporal cliques, the spatial organizing placing the story episodes into a spatial organization so that any first story episode in the temporal clique does not clash with any other story episode in the temporal clique when the story episodes in the temporal clique are played, a clash being an interference between one or more played story episodes;

placing each story in a node of a directed graph, each of the nodes being a story node, the story nodes having relationships among one another established by one or more arcs of the directed graph, the relationships defined by asynchronous conditions, and each of the story nodes on the directed graph capable of being played if the asynchronous conditions of an arc establishing relationships with the story node are satisfied.

9. A method of organizing and playing stories in a hyperstory, as in claim 8, where the directed graph is a petri net.

10. A method of organizing and playing stories, as in claim 8, where the relationship between two or more nodes is a branch relationship where control is passed from a story in a first node to a story in a second node.

11. A method of organizing and playing stories, as in claim 8, where the relationship between two or more of the story nodes is a call relationship where control is passed from a first story in a first story node to a second story in a second story node and then back to the first story after part or all of the second story finishes.

12. A method of organizing and playing stories, as in claim 8, where one or more conditions is enabled by a button.

13. A method of organizing and playing stories, as in claim 8, where the conditions are satisfied to change playing of the stories in the nodes during the execution of the method.

14. A method of organizing and playing stories, as in claim 8, where controls enable conditions that are used to browse the hyperstory.

15. A method of organizing and playing stories, as in claim 8, where the stories are played to educate a user.

16. A method of playing a multimedia story, comprising the following steps:

selecting one or more multimedia segments from a plurality of segments, the selected segments being episodes of the story, the multimedia segments having one or more sensory qualities when played on a multimedia computer system and the multimedia segments being stored on a memory of the computer system;

scheduling the story episodes in a schedule sequence that defines the story;

grouping the story episodes into one or more temporal cliques, each of one or more first episodes in each temporal clique overlapping one or more of the other episodes in the temporal clique by having a part of a first duration of the first episode scheduled at a time in the schedule sequence that is the same as a part of a duration of at least one, but not all, of the other episodes in the temporal clique, at least one of the first durations not starting at the same time as the duration of at least one or more of the other episodes in the temporal clique;

spatially organizing the episodes in each temporal clique independently of the episodes in other temporal cliques, the spatial organization placing the episodes in the temporal clique in a spatial organization so that the first episode does not clash with any of the other episodes in the temporal clique when the episodes in the temporal clique are played, a clash occurring when a first multimedia segment plays at the same time as one or more other multimedia segments and the first multimedia segment interferes with one or more of the other multimedia segments; and

playing the episodes of the story in time according to the schedule and in space according to the spatial organization of the temporal cliques.

17. An apparatus for playing multimedia stories, comprising:

a computer means having a memory means for storing a plurality of multimedia segments;

a means for selecting a subset of multimedia segments, the subset of segments being episodes in a story, the multimedia segments having one or more sensory qualities when played on the computer means;

a temporal layout means for scheduling the story episodes in time to define the story;

a means for organizing the story episodes into temporal cliques, each temporal clique containing one or more first episodes that overlap one or more other episodes in their respective temporal clique, the overlap being a part of a first duration of one or more of the first episodes being scheduled at a time in the temporal layout means that is the same as a part of a duration of at least one, but not all, of the other episodes in the temporal clique, at least one of the first durations not starting at the same time as the duration of at least one or more of the other episodes in the temporal clique;

a means for spatially organizing the story episodes of each temporal clique independently of the spatial organization of the story episodes in other temporal cliques, the spatially organizing placing the episodes in the temporal clique in a spatial organization so that the first episode does not clash with any of the other episodes in the temporal clique when the episodes in the temporal clique are played, a clash occurring when a first multimedia segment plays at the same time as one or more other multimedia segments and the first multimedia segment interferes with one or more of the other multimedia segments; and

a computer playing means for playing the episodes in time as defined by the temporal layout schedule and in space according to the spatial organization of the temporal cliques.