System for video, audio, and graphic presentation in tandem with video/audio play

• US 6,314,569 B1
• Filed: 11/25/1998
• Issued: 11/06/2001
• Est. Priority Date: 11/25/1998
• Status: Expired due to Term

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Chernock, Richard Steven, Dettori, Paolo, Schaffa, Frank Andre, Seidman, David Israel
• Primary Examiner(s): Faile, Andrew
• Assistant Examiner(s): BROWN, RUEBEN M
• Application Number:
  US09/200,212
• Time in Patent Office: 1,077 Days
• Field of Search: 348/9, 348/10, 348/473, 348/474, 348/476-485, 348/563, 348/564, 348/584, 348/589, 348/590, 348/600, 348/906, 348/1-13, 348/586, 348/588, 455/2-6, 725/37, 725/142, 725/112, 345/328
• US Class Current: 725/37
• CPC Class Codes: H04N 21/431 Generation of visual interfaces for content selection or interaction; Content or additional data rendering
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94

Abstract

A method for displaying an enhanced multimedia presentation including personalized supplementary audio, video, and graphic content selectable by a user and rendered by a receiving device, comprises the steps of: communicating a multimedia presentation file to the receiving device, the multimedia presentation file comprising base multimedia presentation content and, frame-synchronized information including starting frame timing identifier, ending frame timing identifier, starting frame spatial coordinates, ending frame spatial coordinates, and motion vector specifications for describing frame-accurate location, motion and timing of the personalized supplementary audio, video, and graphic content, the frame-synchronized information indicating one or more free areas of the multimedia presentation absent significant base multimedia content; extracting the frame-synchronized information from the multimedia presentation file; retrieving the personalized supplementary content from the receiving device; decoding the personalized supplementary content at a time sufficiently in advance of the starting frame timing identifier; and the receiving device selecting an indicated free area and initiating display of one or more items of the personalized supplementary content at frame-accurate times between the starting frame timing identifier and ending frame timing identifier at the frame coordinates in accordance with the frame-synchronized information.

Claims

1. A method for displaying an enhanced multimedia presentation including personalized supplementary audio, video, and graphic content selectable by a user and rendered by a receiving device, the method comprising:

communicating a multimedia presentation file to said receiving device, said multimedia presentation file comprising base multimedia presentation content and, frame-synchronized information including starting frame timing identifier, ending frame timing identifier, starting frame spatial coordinates, ending frame spatial coordinates, and motion vector specifications for describing frame-accurate location, motion and timing of said personalized supplementary audio, video, and graphic content, said frame-synchronized information indicating one or more free areas of said multimedia presentation absent significant base multimedia content;

extracting said frame-synchronized information from said multimedia presentation file;

retrieving said personalized supplementary audio, video and graphic content from said receiving device;

decoding said personalized supplementary audio, video and graphic content at a time sufficiently in advance of said starting frame timing identifier; and

said receiving device selecting an indicated free area and initiating display of one or more items of said personalized supplementary audio, video and graphic content at frame-accurate times between said starting frame timing identifier and ending frame timing identifier at said frame coordinates in accordance with said frame-synchronized information.

2. The method of claim 1, wherein said supplementary audio, video, and graphic content is stored in said receiving device.

3. The method of claim 2, wherein said supplementary audio, video, and graphic content is communicated with said multimedia presentation base content.

4. The method of claim 3, wherein said frame-synchronized information is determined and embedded in said multimedia presentation file in an authoring step prior to the communication step.

5. The method of claim 4 wherein said frame-synchronized information is allowed to be altered in said receiving device via a user interface.

6. The method of claim 5, wherein said frame-synchronized information further includes:

an identifier for coordination with an video/audio content insertion application, a media type, and, description of a bitmap if said video is to be inserted, and volume level if audio is to be inserted.

7. The method of claim 6, wherein said frame-synchronized information is defined in such a way that displaying of said supplementary audio, video and graphic content will not interfere with viewing of said multimedia presentation base content.

8. The method of claim 7, wherein said frame-synchronized information is defined in frame-synchronized coordination with visible objects in said multimedia presentation base content.

9. The method of claim 8, wherein said frame-synchronized information is defined in such a way that supplementary audio play can be performed without interfering with the sound of said multimedia presentation base content.

10. The method of claim 9, wherein said frame-synchronized information is defined in such a way that supplementary audio content can be introduced in coordination with the audio units of said multimedia presentation base content.

11. The method of claim 10, wherein said frame-synchronized information is used in displaying said supplementary audio, video, and graphic content in such a way as not to interfere with the viewing or hearing of said multimedia presentation base content.

12. The method of claim 11, wherein said frame-synchronized information is used in displaying said supplementary audio, video, and graphic content which are coordinated with base audio, video and graphic content of said multimedia presentation, forming a hybrid of coordinated presentation from the conjunction of said base multimedia presentation content and said supplementary audio, video, and graphic content.

13. The method of claim 12, wherein personalization information is stored in said receiving device via said user interface.

14. The method of claim 13, wherein said personalization information includes:

said viewer'"'"'s name, said viewer'"'"'s age, content restriction for said viewer, text preference, audio replacement enablement switch, video replacement enablement switch, and a pointer to a sprite associated with a viewer.

15. A computer program device readable by a machine, tangibly embodying a program of instructions executable by a machine to perform method steps for displaying an enhanced multimedia presentation including personalized supplementary audio, video, and graphic content selectable by a user and rendered by a receiving device, the method comprising:

communicating a multimedia presentation file to said receiving device, said multimedia presentation file comprising base multimedia presentation content and, frame-synchronized information including starting frame timing identifier, ending frame timing identifier, starting frame spatial coordinates, ending frame spatial coordinates, and motion vector specifications for describing frame-accurate location, motion and timing of said personalized supplementary audio, video, and graphic content, said frame-synchronized information representing free areas of said multimedia presentation absent significant base multimedia content;

extracting said frame-synchronized information from said multimedia presentation file;

retrieving said personalized supplementary audio, video and graphic content from said receiving device;

decoding said personalized supplementary audio, video and graphic content at a time sufficiently in advance of said starting frame timing identifier; and

selecting an indicated free area and initiating display of one or more items of said personalized supplementary audio, video and graphic content at frame-accurate times between said starting frame timing identifier and ending frame timing identifier at said frame coordinates in accordance with said frame-synchronized information.

16. The method of claim 15, wherein said supplementary audio, video, and graphic content is stored in said receiving device.

17. The method of claim 16, wherein said supplementary audio, video, and graphic content is communicated with said multimedia presentation base content.

18. The method of claim 17, wherein said frame-synchronized information is determined in an authoring step prior to the communication step.

19. The method of claim 18, wherein said frame-synchronized information is allowed to be altered in said receiving device via a user interface.

20. The method of claim 19, wherein said frame-synchronized information further includes:

an identifier for coordination with an video/audio content insertion application, a media type, and, description of a bitmap if said video is to be inserted, and volume level if audio is to be inserted.

21. The method of claim 20, wherein said frame-synchronized information is defined in such a way that displaying of said supplementary audio, video and graphic content will not interfere with viewing of said multimedia presentation base content.

22. The method of claim 21, wherein said frame-synchronized information is defined in frame-synchronized coordination with visible objects in said multimedia presentation base content.

23. The method of claim 22, wherein said frame-synchronized information is defined in such a way that supplementary audio play can be performed without interfering with the sound of said multimedia presentation base content.

24. The method of claim 23, wherein said frame-synchronized information is defined in such a way that supplementary audio content can be introduced in coordination with the audio units of said multimedia presentation base content.

25. The method of claim 24, wherein said frame-synchronized information is used in displaying said supplementary audio, video, and graphic content in such a way as not to interfere with the viewing or hearing of said, multimedia presentation base content.

26. The method of claim 25, wherein said frame-synchronized information is used in displaying said supplementary audio, video, and graphic content which are coordinated with base audio, video and graphic content of said multimedia presentation, forming a hybrid of coordinated presentation from the conjunction of said base multimedia presentation content and said supplementary audio, video, and graphic content.

27. The method of claim 26, wherein personalization information is stored in a receiving device via said user interface.

28. The method of claim 27, wherein said personalization information includes:

said viewer'"'"'s name, said viewer'"'"'s age, content restriction for said viewer, text preference, audio replacement enablement switch, video replacement enablement switch, and a pointer to a sprite associated with a viewer.