Systems and methods for implementing modular DOM (Document Object Model)-based multi-modal browsers

• US 20020194388A1
• Filed: 12/04/2001
• Published: 12/19/2002
• Est. Priority Date: 12/04/2000
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hosn, Rafah A., Maes, Stephane H., Raman, Thiruvilwamalai V., Seredi, Ladislav, Kleindienst, Jan, Boloker, David, Jang, Photina Jaeyun, Macek, Tomas
• Granted Patent: US 7,028,306 B2
• Time in Patent Office: Days
• US Class Current: 719/310
• CPC Class Codes: G06F 8/38 for implementing user interfaces

Abstract

Systems and methods for building multi-modal browsers applications and, in particular, to systems and methods for building modular multi-modal browsers using a DOM (Document Object Model) and MVC (Model-View-Controller) framework that enables a user to interact in parallel with the same information via a multiplicity of channels, devices, and/or user interfaces, while presenting a unified, synchronized view of such information across the various channels, devices and/or user interfaces supported by the multi-modal browser. The use of a DOM framework (or specifications similar to DOM) allows existing browsers to be extended without modification of the underling browser code. A multi-modal browser framework is modular and flexible to allow various fat client and thin (distributed) client approaches.

Claims

1. A multi-modal browser, comprising:

a model manager for managing a model comprising a modality-independent representation of an application, and

a plurality of channel-specific controllers, wherein each controller processes and transforms the model to generate a corresponding channel-specific view of the model, wherein the channel-specific views are synchronized by the model manager such that a user interaction in one channel-specific view is reflected in another channel-specific view.

2. The multi-modal browser of claim 1, wherein the model manager updates and maintains a dialog state of the model, wherein a user interaction within a given view will update a dialog state of the model.

3. The multi-modal browser of claim 1, wherein the model manager maintains conversation history and context.

4. The multi-modal browser of claim 1, wherein each view is generated by transforming the modality-independent representation of the model to a modality-specific representation.

5. The multi-modal browser of claim 1, wherein the model is programmed using a single authoring framework.

6. The multi-modal browser of claim 5, wherein the single authoring framework comprises a separate an interaction logic layer that encapsulates interaction logic in a modality-independent framework.

7. A multi-modal browser, comprising:

a plurality of modality-dependent browsers; and

a multi-modal shell for parsing and processing a modality-independent application and managing synchronization of I/O (input/output) events across each view generated by the plurality of modality-dependent browsers, wherein each modality-dependent browser comprises;

an API (application programming interface) for controlling the browser and for managing events; and

a wrapper interface comprising synchronization protocols for supporting synchronization of the browser.

8. The multi-modal browser of claim 7, wherein the API for a modality-dependent browser comprises a DOM (document object model) interface.

9. The multi-modal browser of claim 8, wherein the wrapper interface comprises methods for DOM event filtering.

10. The multi-modal browser of claim 7, wherein the multi-modal shell maintains and updates a dialog state of the application.

11. The multi-modal browser of claim 7, wherein the multi-modal browser comprises a fat client framework.

12. The multi-modal browser of claim 7, wherein the multi-modal browser comprises a distributed framework.

13. The multi-modal browser of claim 7, wherein the multi-modal shell comprises:

a model manager for maintaining a dialog state of the application;

a TAV (transformation/adaption/view preparation) manager for preparing and transforming pages or page snippets; and

a synchronization manager for managing event notifications to the browsers.

14. The multi-modal shell of claim 13, wherein the multi-modal shell comprises a distributed framework.

15. The multi-modal browser of claim 7, wherein the plurality of modality-dependent browsers comprise a WML (wireless markup language) browser and a VoiceXML browser.

16. The multi-modal browser of claim 15, further comprising an audio system for capturing and encoding speech data, and a plurality of speech engines for processing speech data.

18. The WAP multi-modal browser of claim 17, wherein the GUI browser comprises a WML (wireless markup language) browser and wherein the voice browser comprise a VoiceXML browser.

19. The WAP multi-modal browser of claim 17, wherein the GUI browser resides on a local client device and the speech application server is distributed over a network.

20. The WAP multi-modal browser of claim 19, further comprising a communication manager that employs protocols for transporting encoded voice data to the speech engines and protocols for enabling synchronization of the GUI browser.

21. The WAP multi-modal browser of claim 19, wherein the speech application server supports protocols for enabling remote control of the engines for server side speech processing.

22. The WAP multi-modal browser of claim 19, wherein the multi-modal shell is distributed over the network.

17. A WAP (wireless application protocol) multi-modal browser, comprising:

a GUI (graphical user interface) browser comprising a DOM (document object model) interface for controlling the GUI browser and managing DOM and event notification and a wrapper interface for filtering events;

a speech application server comprising;

a voice browser, wherein the voice browser comprises a DOM interface for controlling the voice browser and event notification and a wrapper interface for filtering events;

an audio system for capturing and encoding speech data; and

one or more speech engines for processing speech data; and

a multi-modal shell for parsing and processing a modality-independent application and managing synchronization of I/O (input/output) events between the GUI and voice browsers.