Managing application interactions using distributed modality components

• US 20050138219A1
• Filed: 12/19/2003
• Published: 06/23/2005
• Est. Priority Date: 12/19/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: McCobb, Gerald M., Bou-Ghannam, Akram A.
• Granted Patent: US 7,401,337 B2
• Time in Patent Office: Days
• US Class Current: 710/1
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

A method for managing multimodal interactions can include the step of registering a multitude of modality components with a modality component server, wherein each modality component handles an interface modality for an application. The modality component can be connected to a device. A user interaction can be conveyed from the device to the modality component for processing. Results from the user interaction can be placed on a shared memory are of the modality component server.

Claims

1. A method for managing multimodal interactions comprising the steps of:

registering a plurality of modality components, wherein each modality component handles an interface modality for an application;

connecting said activated modality component to a device; and

conveying a user interaction from the device to the modality component for processing.

2. The method of claim 1, wherein said modality component is registered with a modality component server, said method further comprising the step of:

placing results from said user interaction onto a shared memory area of said modality component server.

3. The method of claim 1, said registering step further comprising the step of:

for each modality component, establishing a list of activation conditions such that at least one operation of the modality component is fired when one of said activation conditions is detected.

4. The method of claim 1, wherein at least one of said plurality of modality components is remotely located from said device.

5. The method of claim 4, wherein said device lacks available resources to locally execute at least one function that is handled by the remotely located modality component.

6. The method of claim 4, wherein at least one of said plurality of modality components is disposed within said device.

7. A modality component server comprising:

a modality activator configured to dynamically activate at least one modality component responsive to an occurrence of an application event; and

a multimodal engine configured to detect an interaction and to responsively initiate an interaction response, wherein said interaction and said interaction response have been specified by a previously registered modality component.

8. The server of claim 7, wherein a plurality of modality components are simultaneously utilized, wherein said plurality of modality components specify a plurality of interactions and associated interaction responses, and wherein said multimodal engine is configured to detect any of said plurality of interactions and to responsively initiate an programmatic action.

9. The server of claim 7, wherein said thin client lacks sufficient resources to locally execute one or more functions of said multimodal application.

10. The server of claim 7, wherein said multimodal engine is further configured to manage multimodal interactions involving multiple modality components.

11. The server of claim 7, said multimodal engine further comprising:

a shared memory area;

a list of activation conditions; and

an inference engine configured to run at least one activation condition from said list of activation conditions based on a current state of the shared memory area.

12. A machine-readable storage having stored thereon, a computer program having a plurality of code sections, said code sections executable by a machine for causing the machine to perform the steps of:

registering a plurality of modality components, wherein each modality component handles an interface modality for an application;

connecting said activated modality component to a device; and

conveying a user interaction from the device to the modality component for processing.

13. The machine-readable storage of claim 12, wherein said modality component is registered with a modality component server, said method further comprising the step of:

placing results from said user interaction onto a shared memory area of said modality component server.

14. The machine-readable storage of claim 12, said registering step further comprising the step of:

for each modality component, establishing a list of activation conditions such that at least one operation of the modality component is fired when one of said activation conditions is detected.

15. The machine-readable storage of claim 12, wherein at least one of said plurality of modality components is remotely located from said device.

16. The machine-readable storage of claim 15, wherein said device lacks available resources to locally execute at least one function that is handled by the remotely located modality component.

17. The machine-readable storage of claim 15, wherein at least one of said plurality of modality components is disposed within said device.

18. A system for managing multimodal interactions comprising:

means for registering a plurality of modality components, wherein each modality component handles an interface modality for an application;

means for connecting said activated modality component to a device; and

means for conveying a user interaction from the device to the modality component for processing.