Systems and methods for implementing modular DOM (Document Object Model)-based multi-modal browsers

• US 7,028,306 B2
• Filed: 12/04/2001
• Issued: 04/11/2006
• Est. Priority Date: 12/04/2000
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hosn, Rafah A., Maes, Stephane H., Raman, Thiruvilwamalai V., Kleindienst, Jan, Seredi, Ladislav, Boloker, David, Jang, Photina Jaeyun, Macek, Tomas
• Primary Examiner(s): An, Meng-Al T.
• Assistant Examiner(s): Truong, LeChi
• Application Number:
  US10/007,092
• Publication Number:
  US 20020194388A1
• Time in Patent Office: 1,589 Days
• Field of Search: 719/318, 719/310
• US Class Current: 719/310
• CPC Class Codes: G06F 8/38 for implementing user interfaces

Abstract

Systems and methods for building multi-modal browsers applications and, in particular, to systems and methods for building modular multi-modal browsers using a DOM (Document Object Model) and MVC (Model-View-Controller) framework that enables a user to interact in parallel with the same information via a multiplicity of channels, devices, and/or user interfaces, while presenting a unified, synchronized view of such information across the various channels, devices and/or user interfaces supported by the multi-modal browser. The use of a DOM framework (or specifications similar to DOM) allows existing browsers to be extended without modification of the underling browser code. A multi-modal browser framework is modular and flexible to allow various fat client and thin (distributed) client approaches.

Claims

1. A multi-modal browser, comprising:

a plurality of modality-specific browsers, and

a multi-modal shell that parses and processes a modality-independent representation of an application to generate modality-specific representations of the application which are rendered by corresponding modality-specific browsers to generate modality-specific views of the application, and to manage synchronization of I/O (input/output) events across each modality-specific view,

wherein each modality-specific browser comprises;

an API (application programming interface) that enables the multi-model shell to control the browser and access events; and

a wrapper interface that support synchronization between the modality-specific views by event filtering and exhchanging synchronization information between the multi-modal shell and the browser,

wherein the multi-modal shell comprises;

a model manager for maintaining a dialog state of the application;

a TAV (transformation/adaption/view preparation) manager for preparing and transforming pages or page snippets; and

a synchronization manager for managing event notifications to the browsers.

2. The multi-modal browser of claim 1, wherein the API for a modality-specific browser comprises a DOM (document object model) interface.

3. The multi-modal browser of claim 2, wherein the wrapper interface comprises methods for DOM event filtering.

4. The multi-modal browser of claim 1, wherein the multi-modal shell maintains and updates a dialog state of the application.

5. The multi-modal browser of claim 1, wherein the multi-modal browser comprises a fat client framework.

6. The multi-modal browser of claim 1, wherein the multi-modal browser comprises a distributed framework.

7. The multi-modal shell of claim 1, wherein the multi-modal shell comprises a distributed framework.

8. The multi-modal browser of claim 1, wherein the plurality of modality-specific browsers comprise a WML (wireless markup language) browser and a VoiceXML browser.

9. The multi-modal browser of claim 8, further comprising an audio system for capturing and encoding speech data, and a plurality of speech engines for processing speech data.

10. A distributed WAP (wireless application protocol) multi-modal browser system, comprising:

a GUI (graphical user interface) browser that resides on a client device, wherein the GUI browser comprises a DOM (document object model) interface for controlling the GUI browser and managing DOM event notifications, and a wrapper interface for filtering events;

a speech application server comprising;

a voice browser, wherein the voice browser comprises a DOM interface for controlling the voice browser and managing DOM event notifications, and a wrapper interface for filtering events; and

an audio system for capturing and encoding speech data; and

one or more

speech engines for processing speech data,

wherein the speech application server is distributed over a network;

a multi-modal shell system for parsing and processing a modality-independent application and managing synchronization of I/O (input/output) events between the GUI and voice browsers; and

a communication manager that employs protocols for transporting encoded voice data to the speech engines and protocols for enabling synchronization of the GUI browser.

11. The WAP multi-modal browser of claim 10, wherein the GUI browser comprises a WML (wireless markup language) browser and wherein the voice browser comprises a VoiceXML browser.

12. The WAP multi-modal browser of claim 10, wherein the speech application server supports protocols for enabling remote control of the engines for server side speech processing.

13. The WAP multi-modal browser of claim 10, wherein the multi-modal shell system is distributed over the network.

14. A multi-modal browser, comprising:

a plurality of modality-dependent browsers, and

a multi-modal shell for parsing and processing a modality-independent application and managing synchronization of I/O (input/output) events across each view generated by the plurality of modality-dependent browsers,

wherein each modality-dependent browser comprises;

an API (application programming interface) for controlling the browser and for managing events; and

an external wrapper interface comprising synchronization protocols for supporting synchronization of the browser, and

wherein the multi-modal shell comprises;

a model manager for maintaining a dialog state of the application;

a TAV (transformation/adaption/view preparation) manager for preparing and transforming pages or page snippets; and

a synchronization manager for managing event notifications to the browsers.