Web Service Support For A Multimodal Client Processing A Multimodal Application

• US 20080249782A1
• Filed: 04/04/2007
• Published: 10/09/2008
• Est. Priority Date: 04/04/2007
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: McCobb, Gerald M., Ativanichayaphong, Soonthorn, Cross, Charles W.
• Granted Patent: US 8,788,620 B2
• Time in Patent Office: Days
• US Class Current: 704/275
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

Web service support for a multimodal client processing a multimodal application, the multimodal client providing an execution environment for the application and operating on a multimodal device supporting multiple modes of user interaction including a voice mode and one or more non-voice modes, the application stored on an application server, includes: receiving, by the server, an application request from the client that specifies the application and device characteristics; determining, by a multimodal adapter of the server, modality requirements for the application; selecting, by the adapter, a modality web service in dependence upon the modality requirements and the characteristics for the device; determining, by the adapter, whether the device supports VoIP in dependence upon the characteristics; providing, by the server, the application to the client; and providing, by the adapter to the client in dependence upon whether the device supports VoIP, access to the modality web service for processing the application.

Claims

1. A computer-implemented method of web service support for a multimodal client processing a multimodal application, the multimodal client providing an execution environment for the multimodal application, the multimodal client operating on a multimodal device supporting multiple modes of user interaction including a voice mode and one or more non-voice modes, the multimodal application stored on an application server, the method including:

receiving, by the application server, an application request from the multimodal client, the application request specifying the multimodal application and characteristics for the multimodal device;

determining, by a multimodal adapter of the application server, modality requirements for the multimodal application, the modality requirements specifying characteristics of a modality web service capable of processing multimodal applications;

selecting, by the multimodal adapter of the application server, a particular modality web service in dependence upon the modality requirements and the characteristics for the multimodal device, including retrieving an interface descriptor for the particular modality web service, the particular modality web service capable of processing portions of the multimodal application that correspond to a particular mode of user interaction;

determining, by the multimodal adapter of the application server, whether the multimodal device supports Voice Over Internet Protocol (‘

VoIP’

) in dependence upon the characteristics for the multimodal device;

providing, by the application server, the multimodal application to the multimodal client; and

providing, by the multimodal adapter of the application server to the multimodal client in dependence upon whether the multimodal device supports VoIP, access to the particular modality web service for processing the multimodal application.

2. The method of claim 1 further comprising:

transmitting, by the multimodal client, the application request to the application server;

receiving, by the multimodal client from the application server in response to the application request, the multimodal application;

receiving, by the multimodal client from application server in response to the application request, access to the particular modality web service; and

executing, by the multimodal client, the multimodal application using the particular modality web service, including providing, to the particular modality web service for processing, the portions of the multimodal application that correspond to the particular mode of user interaction capable of being processed by the particular modality web service.

3. The method of claim 1 wherein:

the multimodal adapter determines that the multimodal device supports VoIP; and

providing access to the particular modality web service further comprises;

obtaining, by the multimodal adapter, data communication endpoint information of the particular modality web service in dependence upon the interface descriptor, and

providing, by the multimodal adapter, the data communication endpoint information to the multimodal client for direct access to the particular modality web service.

4. The method of claim 1 wherein:

the multimodal adapter determines that the multimodal device does not support VoIP; and

providing access to the particular modality web service further comprises;

obtaining, by the multimodal adapter, data communication endpoint information of the particular modality web service in dependence upon the interface descriptor, and

providing, by the multimodal adapter, data communication endpoint information of the multimodal adapter to the multimodal client for indirect access to the particular modality web service through the application server

5. The method of claim 1 wherein retrieving an interface descriptor for the particular modality web service further comprises requesting the interface descriptor for the particular modality web service from a modality locator web service.

6. The method of claim 1 further comprising:

receiving, by a multimodal shell web service, a multimodal interface request from the multimodal client, the multimodal interface request specifying user data for a user of the multimodal device and attributes of the multimodal device;

configuring, by the multimodal shell web service, a multimodal interface in dependence upon the user data and the attributes of the multimodal device; and

providing, by the multimodal shell web service, the multimodal interface to the multimodal client.

7. The method of claim 1 further comprising:

transmitting, by the multimodal client to a multimodal shell web service, a multimodal interface request, the multimodal interface request specifying user data for a user of the multimodal device and attributes of the multimodal device;

receiving, by the multimodal client from the multimodal shell web service, a multimodal user interface in response to the user interface request; and

rendering, by the multimodal client, the multimodal user interface to a user.

8. A system for web service support for a multimodal client processing a multimodal application, the multimodal client providing an execution environment for the multimodal application, the multimodal client operating on a multimodal device supporting multiple modes of user interaction including a voice mode and one or more non-voice modes, the multimodal application stored on an application server, the system comprising one or more computer processors and computer memory operatively coupled to the one or more computer processors, the computer memory having disposed within it computer program instructions capable of:

receiving, by the application server, an application request from the multimodal client, the application request specifying the multimodal application and characteristics for the multimodal device;

determining, by a multimodal adapter of the application server, modality requirements for the multimodal application, the modality requirements specifying characteristics of a modality web service capable of processing multimodal applications;

selecting, by the multimodal adapter of the application server, a particular modality web service in dependence upon the modality requirements and the characteristics for the multimodal device, including retrieving an interface descriptor for the particular modality web service, the particular modality web service capable of processing portions of the multimodal application that correspond to a particular mode of user interaction;

determining, by the multimodal adapter of the application server, whether the multimodal device supports Voice Over Internet Protocol (‘

VoIP’

) in dependence upon the characteristics for the multimodal device;

providing, by the application server, the multimodal application to the multimodal client; and

providing, by the multimodal adapter of the application server to the multimodal client in dependence upon whether the multimodal device supports VoIP, access to the particular modality web service for processing the multimodal application.

9. The system of claim 8 wherein the computer memory also has disposed within it computer program instructions capable of:

transmitting, by the multimodal client, the application request to the application server;

receiving, by the multimodal client from the application server in response to the application request, the multimodal application;

receiving, by the multimodal client from application server in response to the application request, access to the particular modality web service; and

executing, by the multimodal client, the multimodal application using the particular modality web service, including providing, to the particular modality web service for processing, the portions of the multimodal application that correspond to the particular mode of user interaction capable of being processed by the particular modality web service.

10. The system of claim 8 wherein:

the multimodal adapter determines that the multimodal device supports VoIP; and

providing access to the particular modality web service further comprises;

obtaining, by the multimodal adapter, data communication endpoint information of the particular modality web service in dependence upon the interface descriptor, and

providing, by the multimodal adapter, the data communication endpoint information to the multimodal client for direct access to the particular modality web service.

11. The system of claim 8 wherein:

the multimodal adapter determines that the multimodal device does not support VoIP; and

providing access to the particular modality web service further comprises;

obtaining, by the multimodal adapter, data communication endpoint information of the particular modality web service in dependence upon the interface descriptor, and

providing, by the multimodal adapter, data communication endpoint information of the multimodal adapter to the multimodal client for indirect access to the particular modality web service through the application server.

12. The system of claim 8 wherein the computer memory also has disposed within it computer program instructions capable of:

receiving, by a multimodal shell web service, a multimodal interface request from the multimodal client, the multimodal interface request specifying user data for a user of the multimodal device and attributes of the multimodal device;

configuring, by the multimodal shell web service, a multimodal interface in dependence upon the user data and the attributes of the multimodal device; and

providing, by the multimodal shell web service, the multimodal interface to the multimodal client.

13. The system of claim 8 wherein the computer memory also has disposed within it computer program instructions capable of:

transmitting, by the multimodal client to a multimodal shell web service, a multimodal interface request, the multimodal interface request specifying user data for a user of the multimodal device and attributes of the multimodal device;

receiving, by the multimodal client from the multimodal shell web service, a multimodal user interface in response to the user interface request; and

rendering, by the multimodal client, the multimodal user interface to a user.

14. A computer program product for web service support for a multimodal client processing a multimodal application, the multimodal client providing an execution environment for the multimodal application, the multimodal client operating on a multimodal device supporting multiple modes of user interaction including a voice mode and one or more non-voice modes, the multimodal application stored on an application server, the computer program product disposed upon a recordable medium, the computer program product comprising computer program instructions capable of:

receiving, by the application server, an application request from the multimodal client, the application request specifying the multimodal application and characteristics for the multimodal device;

determining, by a multimodal adapter of the application server, modality requirements for the multimodal application, the modality requirements specifying characteristics of a modality web service capable of processing multimodal applications;

selecting, by the multimodal adapter of the application server, a particular modality web service in dependence upon the modality requirements and the characteristics for the multimodal device, including retrieving an interface descriptor for the particular modality web service, the particular modality web service capable of processing portions of the multimodal application that correspond to a particular mode of user interaction;

determining, by the multimodal adapter of the application server, whether the multimodal device supports Voice Over Internet Protocol (‘

VoIP’

) in dependence upon the characteristics for the multimodal device;

providing, by the application server, the multimodal application to the multimodal client; and

providing, by the multimodal adapter of the application server to the multimodal client in dependence upon whether the multimodal device supports VoIP, access to the particular modality web service for processing the multimodal application.

15. The computer program product of claim 14 further comprising computer program instructions capable of:

transmitting, by the multimodal client, the application request to the application server;

receiving, by the multimodal client from the application server in response to the application request, the multimodal application;

receiving, by the multimodal client from application server in response to the application request, access to the particular modality web service; and

executing, by the multimodal client, the multimodal application using the particular modality web service, including providing, to the particular modality web service for processing, the portions of the multimodal application that correspond to the particular mode of user interaction capable of being processed by the particular modality web service.

16. The computer program product of claim 14 wherein:

the multimodal adapter determines that the multimodal device supports VoIP; and

providing access to the particular modality web service further comprises;

obtaining, by the multimodal adapter, data communication endpoint information of the particular modality web service in dependence upon the interface descriptor, and providing, by the multimodal adapter, the data communication endpoint information to the multimodal client for direct access to the particular modality web service.

17. The computer program product of claim 14 wherein:

the multimodal adapter determines that the multimodal device does not support VoIP; and

providing access to the particular modality web service further comprises;

obtaining, by the multimodal adapter, data communication endpoint information of the particular modality web service in dependence upon the interface descriptor, and

providing, by the multimodal adapter, data communication endpoint information of the multimodal adapter to the multimodal client for indirect access to the particular modality web service through the application server.

18. The computer program product of claim 14 wherein retrieving an interface descriptor for the particular modality web service further comprises requesting the interface descriptor for the particular modality web service from a modality locator web service.

19. The computer program product of claim 14 further comprising computer program instructions capable of:

receiving, by a multimodal shell web service, a multimodal interface request from the multimodal client, the multimodal interface request specifying user data for a user of the multimodal device and attributes of the multimodal device;

configuring, by the multimodal shell web service, a multimodal interface in dependence upon the user data and the attributes of the multimodal device; and

providing, by the multimodal shell web service, the multimodal interface to the multimodal client.

20. The computer program product of claim 14 further comprising computer program instructions capable of:

transmitting, by the multimodal client to a multimodal shell web service, a multimodal interface request, the multimodal interface request specifying user data for a user of the multimodal device and attributes of the multimodal device;

receiving, by the multimodal client from the multimodal shell web service, a multimodal user interface in response to the user interface request; and

rendering, by the multimodal client, the multimodal user interface to a user.