Encapsulated document and format system

• US 7,284,187 B1
• Filed: 05/30/1997
• Issued: 10/16/2007
• Est. Priority Date: 05/30/1997
• Status: Expired due to Fees

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: AOL LLC (Apollo Global Management, Inc.)
• Inventors: Corboy, David
• Primary Examiner(s): HUYNH, CONG LAC T
• Application Number:
  US08/866,857
• Time in Patent Office: 3,791 Days
• Field of Search: 707513-515, 707/520, 707/526, 707100-104, 707/512, 345/302, 715/500.1, 715/723, 715/726, 715/722, 715/720, 715/716
• US Class Current: 715/203
• CPC Class Codes: G06F 16/4393 Multimedia presentations, e.g. slide shows, multimedia albums

Abstract

Methods for producing and multiplexing a file format, as well as structures for a hierarchical file format and data file, are provided. The data file may include data that is divided in a hierarchical manner, including a highest level document portion that supports all lower level portions of the data file. The hierarchical data file forms a multimedia document that can be displayed on a computer display with accompanying audio. The multimedia document may include data in a variety of file formats, including image data, sound data, textual data, and video data. At least some of the data is preferably multiplexed in the data file, so that the multiplexed data can be progressively played and displayed as it is downloaded by a computer. Data that cannot be progressively played need not be multiplexed in the data file and can be located in an area of the data file separate from the multiplexed data.

Claims

1. A method of streaming multimedia objects encapsulated into a multimedia document, the method comprising:

receiving author specification of multimedia content and choreography information that indicates a temporal order for rendering the multimedia content, the multimedia content being defined by at least first, second, and third multimedia objects and the choreography information including a relative sequence specified by the author for rendering the first, second, and third multimedia objects, the relative sequence indicating that the first and second multimedia objects be rendered progressively together over a common time period and the third multimedia object be rendered subsequent to the rendering of the first and second multimedia objects;

interleaving data slices of the first and second multimedia objects with each other and adding the interleaved data slices to a multimedia document;

adding data slices of the third multimedia object to the multimedia document without interleaving the data slices of the third multimedia object with data slices of other objects in the multimedia document; and

streaming the multimedia document to a recipient for rendering according to the choreography information, such that the data slices of the first and second multimedia objects are progressively rendered together during the common time period before all data slices of the first and second multimedia objects are received and the data slices of the third multimedia object are progressively rendered after the rendering of the first and second multimedia objects and only after a sufficient amount of the data slices of the third multimedia object are received to enable rendering of the third multimedia object.

2. The method of claim 1 wherein the data slices of the first, second, and third multimedia objects are progressively rendered substantially together.

3. The method of claim 1 wherein all of the data slices of the third multimedia object are progressively rendered before any of the data slices of the first and second multimedia object are progressively rendered.

4. The method of claim 1 wherein the data slices of the third multimedia object are rendered after all of the data slices of the first and second multimedia objects are rendered.

5. The method of claim 1 wherein the data slices of the first and second multimedia objects can be progressively rendered independent of bandwidth without degradation.

6. The method of claim 1 wherein the data slices of the third multimedia object cannot be progressively rendered independent of bandwidth without degradation.

7. The method of claim 1 wherein the data slices of the third multimedia object are rendered only after all of the data slices of the third multimedia object are received.

8. The method of claim 1 wherein the choreography information comprises an indication of the author'"'"'s intent that the first, second, and third objects be rendered substantially together.

9. The method of claim 8 wherein the rendering of the first, second, and third objects is delayed until all of the data slices of the third object are received by the recipient.

10. The method of claim 1 wherein the first object comprises a text file.

11. The method of claim 1 wherein the first object comprises an image file.

12. The method of claim 1 wherein the second object comprises a text file.

13. The method of claim 1 wherein the second object comprises an image file.

14. The method of claim 1 wherein the third object comprises a sound file.

15. The method of claim 1 wherein the third object comprises a video file.

16. The method of claim 1 further comprising:

detecting an object type of a detected object that contains at least a portion of the multimedia content; and

determining, based on the detected object type, whether to interleave data slices of the detected object with data slices of another object of the multimedia document or whether to add the data slices of the detected object to the multimedia document without interleaving the data slices of the detected object with data slices of other objects of the multimedia document.

17. The method of claim 16 wherein the detected object is one of the first and second objects.

18. The method of claim 16 wherein the detected object is the third object.

19. The method of claim 1 further comprising rendering the multimedia document in a window on a computer display at the recipient.

20. The method of claim 19 wherein rendering the multimedia document further comprises:

creating an exclusionary area within the window; and

locating an item within the exclusionary area, the item being selected from a group of objects including a framed image, a slide show, framed text, sound data, a separator, or a hyperlink.

21. The method of claim 1 wherein the multimedia content includes splash image data defining a splash image, the method further comprising locating the splash image data within the multimedia document such that the splash image is rendered on a computer display at the recipient as the splash image data is received by a receiver coupled to the computer display.

22. The method of claim 1 further comprising providing each object with an address indicating a player that plays the object.

23. The method of claim 1, further comprising compressing data for at least one of the objects.

24. The method of claim 1 further comprising:

creating an unknown object in the multimedia document; and

locating player data within the unknown object defining a player that plays the unknown object.

25. The method of claim 1 wherein two or more of the objects have at least one common attribute, including at least one of a command for perception of the objects, an ability to pass and receive a message, and an ability to supply and retrieve the data embodied in the objects.

26. The method of claim 1 wherein the multimedia document forms a code segment that receives image information;

and wherein the image information is used to construct an image frame for a framed image that is part of the multimedia document.

27. The method of claim 26 wherein the framed image has an image data format;

and wherein a decoder determines the image data format and encapsulates the framed image with the image frame.

28. The method of claim 1 wherein the choreography information further comprises:

a header;

an object archive for storing information about one or more of the objects, the object archive including information about the relationship of the objects with the document; and

a multiplex section including data for the objects in the document.

29. The method of claim 28 wherein the multiplex section further includes:

an object number counter indicating the number of objects;

a plurality of object descriptions, each object description describing a corresponding one of the objects; and

a choreography group providing information about a first group of objects.

30. The method of claim 29 wherein the choreography group further comprises:

a group object counter indicating the number of objects in the choreography group;

size and type data for each object;

header data; and

the data slices of objects that are interleaved together or the data slices of an object that is not interleaved with other objects.

31. The method of claim 29 further comprising a non-multiplex section following the multiplex section, the non-multiplex section including one or more separate objects that are not played by a player as the separate object files are received by a receiver.

32. The method of claim 1 in which the temporal order is independent of a recipient input.

33. The method of claim 1 in which the temporal order is independent of a recipient hardware configuration.

34. The method of claim 1 in which the temporal order is independent of a recipient software configuration.

35. The method of claim 1, wherein the media content is rendered independent of an author-specified bandwidth to be used to send the multimedia document.

36. A computer implemented device for streaming multimedia objects encapsulated into a multimedia document, the device comprising instructions for:

receiving author specification of multimedia content and choreography information that indicates a temporal order for rendering the multimedia content, the multimedia content being defined by at least first, second, and third multimedia objects and the choreography information including a relative sequence specified by the author for rendering the first, second, and third multimedia objects, the relative sequence indicating that the first and second multimedia objects be rendered progressively together over a common time period and the third multimedia object be rendered subsequent to the rendering of the first and second multimedia objects;

interleaving data slices of the first and second multimedia objects with each other and adding the interleaved data slices to a multimedia document;

adding data slices of the third multimedia object to the multimedia document without interleaving the data slices of the third multimedia object with data slices of other objects in the multimedia document; and

streaming the multimedia document to a recipient for rendering according to the choreography information, such that the data slices of the first and second multimedia objects are progressively rendered together during the common time period before all data slices of the first and second multimedia object are received and the data slices of the third multimedia object are progressively rendered after the rendering of the first and second multimedia objects and only after a sufficient amount of the data slices of the third multimedia object are received to enable rendering of the third multimedia object.

37. The device of claim 36 further comprising instructions for progressively rendering the data slices of the first, second, and third multimedia objects substantially together.

38. The device of claim 36 further comprising instructions for progressively rendering all of the data slices of the third multimedia object before progressively rendering any of the data slices of the first and second multimedia objects.

39. The device of claim 36 further comprising instructions for progressively rendering the data slices of the third multimedia object after progressively rendering all of the data slices of the first and second multimedia objects.

40. The device of claim 36 wherein the data slices of the first and second multimedia objects can be progressively rendered independent of bandwidth without degradation.

41. The device of claim 36 wherein the data slices of the third multimedia object cannot be progressively rendered independent of bandwidth without degradation.

42. The device of claim 36 further comprising instructions for rendering the data slices of the third multimedia object only after all of the data slices of the third multimedia object are received.

43. The device of claim 36 further comprising instructions for:

detecting an object type of a detected object that contains at least a portion of the multimedia content; and

determining, based on the detected object type, whether to interleave data slices of the detected object with data slices of another object of the multimedia document or whether to add the data slices of the detected object to the multimedia document without interleaving the data slices of the detected object with data slices of other objects of the multimedia document.

44. The device of claim 36 further comprising instructions for rendering the media content independent of an author-specified bandwidth to be used to send the multimedia document.

45. A method of streaming multimedia objects encapsulated into a multimedia document, the method comprising:

receiving author specification of multimedia content and choreography information that indicates a temporal order for rendering the multimedia content, the multimedia content being defined by at least first, second, and third multimedia;

interleaving data slices of the first and second multimedia objects with each other and adding the interleaved data slices to a multimedia document;

adding data slices of the third multimedia object to the multimedia document without interleaving the data slices of the third multimedia object with data slices of other objects in the multimedia document;

streaming the multimedia document to a recipient for rendering according to the choreography information, such that the data slices of the first and second multimedia objects are progressively rendered before all data slices of the first and second multimedia objects are received and the data slices of the third multimedia object are progressively rendered only after a sufficient amount of the data slices of the third multimedia object are received to enable rendering of the third multimedia object;

detecting an object of a detected object that contains at least a portion of the multimedia content; and

determining, based on the detected object type, whether to interleave data slices of the detected object with data slices of another object of the multimedia document or whether to add the data slices of the detected object to the multimedia document without interleaving the data slices of the detected object with data slices of other objects of the multimedia document.

46. The method of claim 45 wherein the detected object is one of the first and second objects.

47. The method of claim 45 wherein the detected object is the third object.