INFERRING SWITCHING CONDITIONS FOR SWITCHING BETWEEN MODALITIES IN A SPEECH APPLICATION ENVIRONMENT EXTENDED FOR INTERACTIVE TEXT EXCHANGES

• US 20110270613A1
• Filed: 07/08/2011
• Published: 11/03/2011
• Est. Priority Date: 12/19/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Mandalia, Baiju D., Moore, Victor S., Nusbickel, Wendi L., Da Palma, William V.
• Granted Patent: US 8,239,204 B2
• Time in Patent Office: Days
• US Class Current: 704/260
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

The disclosed solution includes a method for dynamically switching modalities based upon inferred conditions in a dialogue session involving a speech application. The method establishes a dialogue session between a user and the speech application. During the dialogue session, the user interacts using an original modality and a second modality. The speech application interacts using a speech modality only. A set of conditions indicative of interaction problems using the original modality can be inferred. Responsive to the inferring step, the original modality can be changed to the second modality. A modality transition to the second modality can be transparent the speech application and can occur without interrupting the dialogue session. The original modality and the second modality can be different modalities; one including a text exchange modality and another including a speech modality.

Claims

1-20. -20. (canceled)

21. A method for allowing multimodal communication with a speech-enabled application executing on an application server during a communication session with a user, comprising:

with a device other than the application server, receiving voice-based input via a voice input channel and communicating first information corresponding to the voice-based input to the application server for processing by the speech-enabled application during and without interrupting the communication session with the user, and receiving text-based input via a text input channel and communicating second information corresponding to the text based input to the application server for processing by the speech-enabled application during and without interrupting the communication session with the user.

22. The method of claim 21, further comprising:

communicating the first information and the second information from the device to the application server over the same communication channel.

23. The method of claim 22, further comprising:

communicating the first information and the second information from the device to the application server in the same format.

24. The method of claim 21, further comprising:

communicating the first information and the second information from the device to the application server in the same format.

25. The method of claim 21, wherein the device comprises a voice server.

26. The method of claim 21, further comprising:

with a second device other than the application server, selectively providing input from a text exchange client interface and a voice interface to the text input channel and the voice input channel, respectively.

27. The method of claim 26, further comprising:

with the second device, in response to detecting a potential problem with communications from one of the text exchange client interface and the voice interface, providing input from the other of the text exchange client interface and the voice interface to the corresponding one of the text input channel and the voice input channel.

28. A system multi-modal communication system, comprising:

an application server configured to execute a speech-enabled application during a communication session with a user; and

a computer, other than the application server, configured to receive voice-based input via a voice input channel and to communicate first information corresponding to the voice-based input to the application server for processing by the speech-enabled application during and without interrupting the communication session with the user, and further configured to receive text-based input via a text input channel and to communicate second information corresponding to the text based input to the application server for processing by the speech-enabled application during and without interrupting the communication session with the user.

29. The system of claim 28, wherein the computer is further configured to communicate the first information and the second information to the application server over the same communication channel.

30. The system of claim 29, wherein the computer is further configured to communicate the first information and the second information to the application server in the same format.

31. The system of claim 28, wherein the computer is further configured to communicate the first information and the second information from the device to the application server in the same format.

32. The system of claim 28, wherein the computer comprises a voice server.

33. The system of claim 28, further comprising a second computer, other than the application server, configured to selectively provide input from a text exchange client interface and a voice interface to the text input channel and the voice input channel, respectively.

34. The system of claim 33, wherein the second computer is further configured to, in response to detecting a potential problem with communications from one of the text exchange client interface and the voice interface, provide input from the other of the text exchange client interface and the voice interface to the corresponding one of the text input channel and the voice input channel.

35. A system multi-modal communication system, comprising:

an application server configured to execute a speech-enabled application during a communication session with a user; and

means, other than the application server, for receiving voice-based input via a voice input channel and communicating first information corresponding to the voice-based input to the application server for processing by the speech-enabled application during and without interrupting the communication session with the user, and for receiving text-based input via a text input channel and communicating second information corresponding to the text based input to the application server for processing by the speech-enabled application during and without interrupting the communication session with a user.