System and method for providing multi-modal interactive streaming media applications

• US 20040128342A1
• Filed: 12/31/2002
• Published: 07/01/2004
• Est. Priority Date: 12/31/2002
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Maes, Stephane H., Ramaswamy, Ganesh N.
• Application Number:
  US10/335,039
• Publication Number:
  US 20040128342A1
• Time in Patent Office: Days
• US Class Current: 709/200
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/1104 Session initiation protocol [SIP]
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/613 for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

A system and method for generating streamed broadcast or multimedia applications that offer multi-modal interaction with the content of a multimedia presentation. Mechanisms are provided for enhancing multimedia broadcast data by adding and synchronizing low bit rate meta-information which preferably implements a multi-modal user interface. The meta information associated with video or other streamed data provides a synchronized multi-modal description of the possible interaction with the content. The multi-modal interaction is preferably implemented using intent-based interaction pages that are authored using a modality-independent script.

Claims

1. A method for implementing a multimedia application, comprising the steps of:

associating content of a multimedia application to one or more interaction pages; and

presenting a user interface that enables user interactivity with the content of the multimedia application using an associated interaction page.

2. The method of claim 1, wherein the step of associating content of the multimedia application to one or more interaction pages comprises mapping a region of the multimedia application to one or more interaction pages using an image map.

3. The method of claim 2, wherein mapped regions of the multimedia application are logically associated with data models for which user interaction is described using a modality independent, single authoring interaction-based programming paradigm.

4. The method of claim 1, wherein the step of associating content of a multimedia application to one or more interaction pages comprises transmitting low bit rate encoded meta information with a bit stream of the multimedia application.

5. The method of claim 4, wherein the low bit rate encoded meta information is transmitted in band or out of band.

6. The method of claim 4, wherein the encoded meta information describes a user interface that enables a user to control and manipulate streamed content.

7. The method of claim 6, wherein the user interface comprises one of a conversational, multi-modal and multi-channel user interface.

8. The method of claim 1, wherein the interaction pages comprise modality independent interaction pages that describe user interaction using a modality-independent script.

9. The method of claim 8, wherein the modality-independent script is one of declarative, imperative, and a combination thereof.

10. The method of claim 8, comprising the step of transcoding a modality-independent interaction page to a modality-specific interaction page.

11. The method of claim 1, wherein the step of presenting a user interface comprises presenting a multi-modal interface.

12. The method of claim 11, further comprising the step of synchronizing user interaction across all modalities provided by the multi-modal interface.

13. The method of claim 1, comprising the step of using different user agents for rendering multimedia content and an interactive user interface.

14. The method of claim 1, wherein the user interface enables a user to control presentation of the multimedia application.

15. The method of claim 1, wherein the user interface enables a user to control a source of the multimedia application.

16. The method of claim 1, further comprising the step of updating the interaction pages, or fragments thereof, during a multimedia presentation.

17. The method of claim 16, wherein the step of updating comprises selecting interaction pages, or fragments thereof, using a synchronizing application.

18. The method of claim 16, wherein the step of updating comprises using event driven coordination based on events that are thrown during a multimedia presentation.

19. A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform method steps for implementing a multimedia application, the method steps comprising:

associating content of a multimedia application to one or more interaction pages; and

presenting a user interface that enables user interactivity with the content of the multimedia application using an associated interaction page.

20. The program storage device of claim 19, wherein the instructions for associating content of the multimedia application to one or more interaction pages comprise instructions for mapping a region of the multimedia application to one or more interaction pages using an image map.

21. The program storage device of claim 20, wherein mapped regions of the multimedia application are logically associated with data models for which user interaction is described using a modality independent, single authoring interaction-based programming paradigm.

22. The program storage device of claim 19, wherein the instructions for associating content of a multimedia application to one or more interaction pages comprise instructions for transmitting low bit rate encoded meta information with a bit stream of the multimedia application.

23. The method of claim 22, wherein the encoded meta information describes a user interface that enables a user to control and manipulate streamed content.

24. The program storage device of claim 23, wherein the user interface comprises one of a conversational, multi-modal and multi-channel user interface.

25. The program storage device of claim 19, comprising instructions for transcoding a modality-independent interaction page to a modality-specific interaction page.

26. The program storage device of claim 19, wherein the instructions for presenting a user interface comprise instructions for presenting a multi-modal interface.

27. The program storage device of claim 26, further comprising instructions for synchronizing user interaction across all modalities provided by the multi-modal interface.

28. The program storage device of claim 19, wherein different user agents are used for rendering multimedia content and an interactive user interface.

29. The program storage device of claim 19, wherein the user interface enables a user to control presentation of the multimedia application.

30. The program storage device of claim 19, wherein the user interface enables a user to control a source of the multimedia application.

31. The program storage device of claim 19, further comprising instructions for updating the interaction pages, or fragments thereof, during a multimedia presentation.

32. The program storage device of claim 31, wherein the instructions for updating comprise instructions for selecting interaction pages, or fragments thereof, using a synchronizing application.

33. The program storage device of claim 31, wherein the instructions for updating comprise instructions for using event driven coordination based on events that are thrown during a multimedia presentation.

34. A system for enabling interactivity with a multimedia presentation, the system comprising:

a server for associating content of a multimedia application to one or more interaction pages; and

a client for rendering and presenting a user interface that enables user interactivity with the content of the multimedia application using an associated interaction page.

35. The system of claim 34, wherein the server comprises:

a first database comprising a multimedia application and one or more image maps and interaction pages that are associated with the multimedia application; and

a second database for storing mapping information that maps a portion of the multimedia application to an interaction page; and

a coordinator for coordinating interaction pages with the multimedia application.

36. The system of claim 34, wherein the client comprises a multi-modal browser that parses an interaction page and generates a modality-specific script representing the interaction page.

37. The system of claim 34, wherein the client comprises a browser that enables a user to control presentation of the multimedia application or control a source of the multimedia application.

38. The system of claim 34, wherein the client comprises a first user agent for rendering multimedia content and a second user agent for rendering an interactive user interface.