Seamless multimedia branching

• US 6,343,298 B1
• Filed: 06/13/2000
• Issued: 01/29/2002
• Est. Priority Date: 04/03/1997
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Sun, Yi, Nelson, Robert B., Savchenko, Alexander V., Nirkhe, Vivek
• Primary Examiner(s): Homere, Jean R.
• Assistant Examiner(s): ROBINSON, GRETA LEE
• Application Number:
  US09/596,234
• Time in Patent Office: 595 Days
• Field of Search: 707/104, 707/205, 345/302, 709/203, 709/103, 711/4, 711/100
• US Class Current: 1/1
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

Described above are methods of authoring multimedia titles and of storing and delivering multimedia content that implements guaranteed seamless branching on digital media with high seek latency and a fixed upper bound on this latency. Continuous media content is arranged as individual clips on a storage medium. Seamless branches between clips are identified by an author. A method is disclosed to identify individual clips as carrier clips or non-carrier clips to guarantee specified seamless behavior while optimizing bridge memory usage and availability of seamless jumps. Bridge data of a particular target media clip is interleaved or otherwise associated on the storage medium with a carrier clip that is upstream of the target media clip, and delivered along with the upstream media clip. As bridge data are delivered, they are stored in bridge memory, to be used to implement a subsequent seamless branch. After the branch is implemented or after the branch is no longer an option, the bridge data is cleared from the bridge memory. Additional methods are disclosed for dealing with intervening file access and for accounting for no-latency branches that are possible when using some forms of storage devices.

Claims

1. A method of authoring media comprising a plurality of continuous media clips connected by seamless branches, the media clips having bridge data, the method comprising:

assigning certain media clips to be non-carrier clips;

assigning other media clips to be carrier clips; and

scheduling delivery of bridge data for corresponding media clips during immediately upstream carrier clips.

2. A method as recited in claim 1, wherein some of the non-carrier clips are data clips having no outgoing branches, wherein the bridge data of a data clip comprises the entire data clip.

3. A method as recited in claim 1, further comprising a step of disallowing seamless branches during initial parts of carrier clips until bridge data has been delivered.

4. A method as recited in claim 1, wherein some of the branches are no-latency seamless branches, wherein the scheduling step does not include scheduling delivery of bridge data for target clips of no-latency seamless branches.

5. A method as recited in claim 1, further comprising a step of calculating bridge memory requirements based on a particular assignment of carrier and non-carrier clips.

6. A method as recited in claim 1, further comprising the following additional steps:

calculating bridge memory requirements based on a particular assignment of carrier and non-carrier clips; and

reassigning individual media clips to be carrier or non-carrier clips to change bridge memory requirements.

7. A method as recited in claim 1, wherein the branches are from source to target media clips, further comprising the following additional steps:

identifying target bridge sets corresponding to respective media clips, wherein a target bridge set corresponding to a particular media clip comprises (a) target bridge sets of non-carrier clips that are immediate targets of the particular media clip and (b) the bridge data of carrier clips that are immediate targets of the particular media clip;

during delivery of a particular carrier clip, scheduling delivery of its target bridge set.

8. A method as recited in claim 1, wherein said target bridge set corresponding to said particular media clip does not include bridge data of any media clips that are targeted by no-latency branches from said particular media clip.

9. A method as recited in claim 1, further comprising a step of storing bridge data in bridge memory as it is delivered.

10. A method as recited in claim 1, further comprising the following additional steps:

storing bridge data in bridge memory as it is delivered;

clearing the bridge data memory at times corresponding to delivery of the carrier clips.

11. A method as recited in claim 1, wherein the branches are from source to target media clips, further comprising a step of scheduling any bridge data required to begin rendering any target clip of a particular non-carrier source clip for delivery during a carrier clip that is upstream of said particular non-carrier source clip.

12. A method as recited in claim 1, wherein the branches are from source to target media clips, further comprising the following additional steps:

allowing an author to assign non-carrier media clips;

scheduling any bridge data required to begin rendering any target clip of a particular non-carrier source clip for delivery during a carrier clip that is upstream of said particular non-carrier source clip.

13. A method as recited in claim 1, further comprising a step of delivering the media clips from a storage medium that imposes an access latency between media clips.

14. A computer-readable storage medium containing computer-executable instructions for performing the steps of claim 1.

15. A method as recited in claim 1, further comprising a step of interleaving the bridge data for a particular target clip in an upstream carrier clip.

16. A method of authoring media comprising a plurality of continuous media clips, media clips having bridge data, the method comprising:

connecting the media clips with a plurality of seamless branches from source clips to target clips;

assigning select media clips as non-carrier clips and others as carrier clips;

scheduling any bridge data required to begin rendering any target clip of a particular non-carrier source clip for delivery during an immediately upstream carrier clip.

17. A method as recited in claim 16, wherein some of the non-carrier clips are data clips having no outgoing branches, wherein the bridge data of a data clip comprises the whole data clip.

18. A method as recited in claim 16, further comprising a step of disallowing seamless branches during initial parts of media clips that are not non-carrier clips until bridge data has been delivered.

19. A method as recited in claim 16, wherein some of the branches are no-latency seamless branches, wherein the scheduling step does not include scheduling delivery of bridge data for target clips of no-latency seamless branches.

20. A method as recited in claim 16, further comprising a step of calculating bridge memory requirements based on a particular assignment of non-carrier clips.

21. A method as recited in claim 16, further comprising a step of storing bridge data in bridge memory as it is delivered.

22. A computer-readable storage medium containing computer-executable instructions for performing the steps of claim 16.

23. A method as recited in claim 16, further comprising a step of interleaving the bridge data for a particular target clip in an upstream media clip that is not a non-carrier clip.

24. A method of storing a plurality of continuous media clips on a storage medium with an inherent access latency between the media clips, wherein the media clips are to be interconnected during rendering by seamless branches from upstream source media clips to downstream target media clips, the method comprising the following steps:

identifying bridge data for target clips; and

combining the bridge data for a particular target clip in an upstream media clip on the storage medium.

25. A method as recited in claim claim 24, further comprising a step of combining the bridge data for a particular target clip in an immediately upstream media clip on the storage medium.

26. A method as recited in claim claim 24, wherein the combining step comprises interleaving the bridge data for a particular target clip in an upstream media clip on the storage medium.

27. A method as recited in claim claim 24, wherein some of the media clips are data clips having no outgoing branches, and wherein the bridge data of a data clip comprises the entire data clip.

28. A method of rendering a plurality of continuous media clips from a storage medium that imposes an access latency between clips, wherein the clips are to be interconnected during rendering by seamless branches from upstream source clips to downstream target clips, the method comprising:

identifying bridge data for target clips, the bridge data comprising an initial part of a target clip that can be rendered during the storage medium access latency;

combining the bridge data for individual target clips in upstream clips on the storage medium;

reading media clips from the storage medium;

while reading a particular media clip, also reading the bridge data for one or more individual target clips from the storage medium;

buffering the bridge data read from the storage medium; and

rendering the bridge data corresponding to a particular target clip during the access latency when branching to said particular target clip.

29. A method as recited in claim 28, further comprising a step of combining the bridge data for a particular target clip in an immediately upstream source clip.

30. A method as recited in claim 28, wherein the combining step comprises interleaving the bridge data for individual target clips in upstream media clips on the storage medium.

31. A method as recited in claim 28, wherein the buffering step comprises storing the bridge data in bridge data memory other than the storage medium, the method further comprising a step of clearing bridge data from the bridge memory after a branch to the target clip of said bridge data is no longer allowed.

32. A method as recited in claim 28, wherein some of the media clips are data clips having no outgoing branches, and wherein the bridge data of a data clip comprises the entire data clip.

33. A method as recited in claim 28, wherein the combining step comprises interleaving the bridge data for individual target clips in upstream clips on the storage medium.

34. A computer-readable storage medium with an inherent access latency, comprising:

a plurality of continuous media clips stored on the storage medium, wherein the media clips are to be interconnected during rendering by seamless branches from upstream source clips to downstream target clips;

the target clips having bridge data comprising initial parts of the target clips that can be rendered during the storage medium access latency; and

the bridge data for target clips being combined with upstream media clips on the storage medium.

35. A computer-readable storage medium as recited in claim 34, wherein the bridge data for a particular target clip is combined in an immediately upstream media clip on the storage medium.

36. A computer-readable storage medium as recited in claim 34, wherein the bridge data for a particular target clip is interleaved in an upstream media clip on the storage medium.

37. A computer-readable storage medium as recited in claim 34, wherein some of the media clips are data clips having no outgoing branches, and wherein the bridge data of a data clip comprises the entire data clip.

38. An apparatus comprising:

a storage medium including a plurality of executable instructions; and

an execution unit, coupled to the storage medium, to execute at least a subset of the plurality of executable instructions to implement a media authoring tool, wherein the media authoring tool enables a user to define media clips as either carrier or non-carrier clips, and to interleave bridge data for a subsequent carrier or non-carrier clips in an immediately upstream carrier clip to create a plurality of media clips connected by seamless branches.

39. An apparatus as recited in claim 38, wherein the authoring tool calculates bridge memory requirements based on a particular definition of carrier and non-carrier clips.

40. An apparatus as recited in claim 38, wherein the authoring tool calculates bridge memory requirements based on a particular definition of carrier and non-carrier clips, and reassigns individual media clips to be carrier or non-carrier clips to change the bridge memory requirements.

41. An apparatus as recited in claim 38, wherein the branches lead from source clips to target clips and wherein the authoring tool:

identifies a target bridge set corresponding to a particular media clip, the target bridge set comprising (a) target bridge sets of all non-carrier clips that are immediate targets of the particular media clip, and (b) the bridge data of all carrier clips that are immediate targets of the particular media clip; and

identifies the largest target bridge set, wherein the largest target bridge set corresponds to a minimum requirement for bridge memory.

42. An apparatus as recited in claim 38, wherein the authoring tool stores the plurality of continuous media clips on a storage device, for subsequent access and rendering.