Distributed multimodal dialogue system and method

• US 20040034531A1
• Filed: 08/15/2002
• Published: 02/19/2004
• Est. Priority Date: 08/15/2002
• Status: Abandoned Application

• Current Assignee: Avaya Incorporated
• Original Assignee: Avaya Incorporated
• Inventors: Chou, Wu, Ll, Li, Saad, Antoine, Liu, Feng
• Application Number:
  US10/218,608
• Publication Number:
  US 20040034531A1
• Time in Patent Office: Days
• US Class Current: 704/270.1
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/401 wherein the services involve a main real-time session and one or more additional parallel real-time or time sensitive sessions, e.g. white board sharing or spawning of a subconference
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/565 Conversion or adaptation of application format or content adding application control or application functional data H04L67/561
• CPC Class Codes: H04L 67/566 Grouping or aggregating service requests, e.g. for unified processing
• CPC Class Codes: H04L 69/08 Protocols for interworking; Protocol conversion
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

A system and method for providing distributed multimodal interaction are provided. The system is a hybrid multimodal dialogue system that includes one or multiple hybrid constructs to form sequential and joint events in multimodal interaction. It includes an application interface receiving a multimodal interaction request for conducting a multimodal interaction over at least two different modality channels; and at least one hybrid construct communicating with multimodal servers corresponding to the multiple modality channels to execute the multimodal interaction request.

Claims

1. A distributed multimodal interaction system comprising:

an application interface receiving a multimodal interaction request for conducting a multimodal interaction over at least two different modality channels; and

at least one hybrid construct communicating with multimodal servers corresponding to the modality of channels to execute the multimodal interaction request.

2. The system of claim 1, wherein the system is a hybrid voice extensible markup language (VoiceXML) system including one or multiple hybrid constructs.

3. The system of claim 1, wherein the hybrid construct receives responses to the multimodal interaction request from the multiple modality channels, and compiles a joint event response based on the responses from each individual modality, and transmits the joint event response to the application interface to conduct the multimodal interaction.

4. The system of claim 3, wherein the joint event response is compiled in the form of an extensible markup language (XML) page.

5. The system of claim 1, wherein the at least two modality channels include a voice channel, and the system further comprises an interpreter and a web server for processing voice dialogue over the voice channel.

6. The system of claim 1, wherein the hybrid construct includes:

a server page communicating with the application interface or a voice browser;

at least one synchronizing modules distributing the multimodal interaction request to the appropriate multimodal servers over the different modality channels; and

at least one dialogue agent communicating the multimodal interaction request with the appropriate multimodal servers, receiving the responses from the multimodal servers, and delivering the responses to the server page.

7. The system of claim 1, wherein the at least two modality channels include different types of voice dialogue channels.

8. The system of claim 7, wherein the types of voice dialogue channels include a natural language dialogue channel and a finite-state dialogue channel.

9. The system of claim 1, wherein the at least two modality channels include at least two of the following:

voice, e-mail, fax, web-form, and web chat.

10. The system of claim 1, wherein the system conducts the multimodal interaction over at least the two modality channels, simultaneously and in parallel.

11. A method of providing distributed multimodal interaction in a dialogue system, the dialogue system including an application interface and at least one hybrid construct, the method comprising:

receiving, by the application interface, a multimodal interaction request for conducting a multimodal interaction over at least two different modality channels; and

communicating, by the hybrid construct, with multimodal servers corresponding to the modality channels to execute the multimodal interaction request.

12. The method of claim 11, wherein the dialogue system is a hybrid voice extensible markup language (VoiceXML) system with one or multiple hybrid constructs.

13. The method of claim 11, wherein the communicating step includes:

receiving, by the hybrid construct, responses to the multimodal interaction request from the modality channels;

compiling a joint event response based on the responses; and

transmitting the joint event response to the application interface to conduct the multimodal interaction.

14. The method of claim 13, wherein the joint event response is compiled in the form of an extensible markup language (XML) page.

15. The method of claim 11, wherein the at least two modality channels include a voice channel, and the method further comprises processing voice dialogue over the voice channel.

16. The method of claim 11, wherein the communicating step includes:

communicating by a server page with the application interface or a voice browser;

distributing the multimodal interaction request to the appropriate multimodal servers over the modality channels using at least one synchronizing module; and

communicating the multimodal interaction request with the appropriate multimodal servers using at least one dialogue agent, receiving the responses from the multimodal servers, and delivering the responses to the server page.

17. The method of claim 11, wherein the at least two modality channels include different types of voice dialogue channels.

18. The method of claim 17, wherein the types of voice dialogue channels include a natural language dialogue channel and a finite-state dialogue channel.

19. The method of claim 11, wherein the at least two modality channels include at least two of the following:

voice, e-mail, fax, web-form, and web chat.

20. The method of claim 11, wherein the multimodal interaction is conducted over at least the two modality channels, simultaneously and in parallel.

21. A computer program product embodied on computer-readable media, for providing distributed multimodal interaction in a dialogue system, the dialogue system including an application interface and at least one hybrid construct, the computer program product comprising computer-executable instructions for;

receiving, by the application interface, a multimodal interaction request for conducting a multimodal interaction over at least two different modality channels; and

communicating, by the hybrid construct, with multimodal servers corresponding to the modality channels to execute the multimodal interaction request.

22. The computer program product of claim 21, wherein the dialogue system is a hybrid voice extensible markup language (VoiceXML) system with one or multiple hybrid constructs.

23. The computer program product of claim 21, wherein the computer-executable instructions for communicating include computer-executable instructions for:

receiving, by the hybrid construct, responses to the multimodal interaction request from the modality channels;

compiling a joint event response based on the responses; and

transmitting the joint event response to the application interface to conduct the multimodal interaction.

24. The computer program product of claim 23, wherein the joint event response is compiled in the form of an extensible markup language (XML) page.

25. The computer program product of claim 21, wherein the at least two modality channels include a voice channel, and the computer program product further comprises computer-executable instructions for processing voice dialogue over the voice channel.

26. The computer program product of claim 21, wherein the computer-executable instructions for communicating include computer-executable instructions for:

communicating by a server page with the application interface or a voice browser;

distributing the multimodal interaction request to the appropriate multimodal servers over the modality channels using at least one synchronizing module; and

communicating the multimodal interaction request with the appropriate multimodal servers using at least one dialogue agent, receiving the responses from the multimodal servers, and delivering the responses to the server page.

27. The computer program product of claim 21, wherein the at least two modality channels include different types of voice dialogue channels.

28. The computer program product of claim 27, wherein the types of voice dialogue channels include a natural language dialogue channel and a finite-state dialogue channel.

29. The computer program product of claim 21, wherein the at least two modality channels include at least two of the following:

voice, e-mail, fax, web-form, and web chat.

30. The computer program product of claim 21, wherein the multimodal interaction is conducted over at least the two modality channels, simultaneously and in parallel.