Mode-swapping in multi-modal telephonic applications

• US 7,289,606 B2
• Filed: 04/10/2002
• Issued: 10/30/2007
• Est. Priority Date: 10/01/2001
• Status: Expired due to Term

• Current Assignee: CDN Innovations, LLC (IP Investments Group LLC)
• Original Assignee: Kirusa Inc.
• Inventors: Sibal, Sandeep, Mumick, Inderpal Singh
• Primary Examiner(s): Matar, Ahmad F.
• Assistant Examiner(s): LE, KAREN L
• Application Number:
  US10/119,614
• Publication Number:
  US 20030063728A1
• Time in Patent Office: 2,029 Days
• Field of Search: 379/52, 379/93.09, 379/93.17, 379/207.02
• US Class Current: 379/52
• CPC Class Codes: H04M 2203/251 where a voice mode or a visual mode can be used interchangeably
• CPC Class Codes: H04M 2203/253 where a visual mode is used instead of a voice mode
• CPC Class Codes: H04M 2207/18 wireless networks
• CPC Class Codes: H04M 3/493 Interactive information services, e.g. directory enquiries ; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

Multi-modal applications are capable of interacting with a user in a voice mode, a visual mode, and/or other modes of communication. Mode swapping refers to the process of changing from one mode of communication (or one combination of modes) to another. Mode swapping may be initiated by a user, by an application, or by another entity. Mode swapping may be performed “adaptively,” in that an entity monitors the effectiveness of a communication mode and initiates a mode swap if communication is proceeding non-optimally in the current mode. Mode swapping may be performed by dropping a call to a telephone terminal in the currently-open mode and placing a call to that same device in another mode. In one embodiment, logic for mode swapping resides on a gateway proxy that mediates communication between an application and a telephone terminal.

Claims

1. A method of swapping between a first mode of communication and a second mode of communication, wherein an application and a user communicate through a telephone in said first and second modes of communication, the communication being in a first mode, wherein a first browser facilitates communication with application in the first mode, and a second browser facilitates communication with application in the second mode, the method comprising:

generating an instruction to perform a change of mode;

adding the second mode to the communication while maintaining simultaneous use of the first mode of communication, wherein at the instant of adding the second browser, the second browser is not yet synchronized to the state of the first browser;

synchronizing the second browser to a state of the first browser, further comprising the steps of;

maintaining state information of said first browser and said second browser;

receiving said maintained state information of first browser at the second browser; and

initializing the second browser to the state of the first browser;

whereby the first browser and the second browser are not required to be concurrently in the same updated state in all points in time during communication.

2. The method of claim 1, wherein said instruction is generated by the application.

3. The method of claim 1, wherein said instruction is generated by the user.

4. The method of claim 1, wherein said instruction is generated by an entity different from the application and different from the user.

5. The method of claim 1, wherein one of the modes comprises a data mode which enables the user to receive input visually and to provide input tactilely, and wherein the other of the modes comprises a voice mode which enables the user to receive input audibly and to provide audible input.

6. The method of claim 1, wherein the act of adding the second mode comprises placing a call to the telephone.

7. The method of claim 1, further comprising:

dropping an existing call to the telephone while maintaining communication between the telephone and the application in at least one of the modes.

8. The method of claim 1, wherein the first mode comprises a voice mode, and wherein the method further comprises:

detecting that there is a difficulty in deciphering the user'"'"'s speech; and

generating the instruction in response to said detecting step.

9. A gateway proxy for mode swapping between a first mode of communication and a second mode of communication between a telephone operated by a user and an application, wherein a first browser is used for conducting said first mode of communication and a second browser is used for conducting said second mode of communication, said gateway proxy comprising:

a swapping logic for opening and closing said first mode connection and said second mode connection in accordance with a mode instruction, wherein said mode instruction can cause to be enabled any of the following;

a second mode only connection;

a first mode only connection; and

a simultaneous first mode and second mode connection, wherein the first mode and second mode connections are synchronized upon in addition of one of the (i) second mode connection and (ii) the first mode connection to the simultaneous second mode and first mode connection;

a state repository for maintaining information about state of navigation in the application;

an adaptive mode swapping logic for analyzing current conditions and determining whether to perform a mode swap based on said conditions.

10. The gateway proxy of claim 9, wherein said swapping logic receives said instruction from said telephone.

11. The gateway proxy of claim 9, wherein said swapping logic receives said instruction from said application.

12. The gateway proxy of claim 9, wherein said swapping logic opens at least one of the connections by placing a call to said telephone, and wherein said logic closes at least one of the connections by dropping a call to said telephone.

13. The gateway proxy of claim 9, wherein said swapping logic is adapted to open and close said voice connection separately for input and output, wherein said logic receives an instruction to discontinue output to said telephone, and wherein said logic closes the output of said voice connection in response to the received instruction.

14. A method of interacting with a device, wherein said device is in communication with an application, and wherein said device is capable of communicating in both:

(1) a voice mode which receives audio input and provides audio output, and (2) a visual mode which receives non-audible input and provides visual output, wherein a voice browser facilitates communication with said application in the voice mode, and wherein a visual browser facilitates communication with the application in the visual mode, the method comprising;

interacting with the device in the voice mode, including receiving information from the device in the voice mode, but without sending information to the device in voice the mode;

adding the visual mode to the communication while maintaining simultaneous use of the voice mode from the communication;

interacting with the device in visual mode concurrently with voice mode wherein data is received in the visual mode concurrently with the audio in the voice mode; and

upon adding the visual mode to the communication, synchronizing the visual browser to a state of voice browser.

15. The method of claim 14, further comprising:

discontinuing the sending of information to the device in voice mode.

16. The method of claim 15, wherein the discontinuance is in response to a user instruction.

17. The method of claim 15, further comprising:

augmenting the interaction with the device in visual mode to include information that would be sent to the device in voice mode if the sending of information to the device in voice mode had not been discontinued.

18. A method of performing a communication between an application and a user using a mobile telephone having multiple communication modes, wherein a voice browser facilitates communication with said application in the voice mode, and wherein a visual browser facilitates communication with the application in the visual mode, the method comprising:

receiving an instruction from either said application or said user to perform a change of communication mode while in the voice mode;

adding the visual mode to the communication while keeping the voice mode and the visual mode active so as to permit simultaneous communication between the mobile telephone and the application in the voice mode and the visual mode, wherein the voice mode enables the user to receive and transmit audible information and wherein visual mode enables the user to perform at least one of view received information using the visual browser and respond to the received information; and

synchronizing the visual browser to a state of the voice browser, further comprising the steps of;

maintaining state information of said voice browser and said visual browser;

receiving said maintained state information of voice browser at the visual browser; and

initializing the visual browser to the state of the voice browser;

whereby the first browser and the second browser are not required to be concurrently in the same updated state in all points in time during communication.

19. The method of claim 18, wherein receiving an instruction from at least one of a user and an application comprises receiving an instruction from at least one of a gateway and an application server.

20. The method of claim 18, further comprising:

synchronizing the voice browser to the state of visual browser, wherein the visual browser enables interaction with the audible information in the voice mode.

21. The method of claim 18, wherein the instruction is received in response to a difficulty in deciphering the user'"'"'s speech.

22. A system for mode swapping between a visual mode of communication and a voice mode of communication between a telephone operated by a user and an application, wherein a visual browser conducts said visual mode of communication and a voice browser conducts said voice mode of communication, said system comprising:

a visual display rendering output from said visual browser;

a keypad inputting data to said voice browser;

a speaker rendering output from said voice browser and the voice browser accepting inputs from a microphone;

a gateway proxy, further comprising;

a swapping logic for opening and closing said visual mode connection and said voice mode connection in accordance with a mode instruction, wherein said mode instruction can enable any of the following;

a voice mode only connection;

a visual mode only connection; and

a visual first mode and voice mode connection, wherein the visual mode and voice mode connections are synchronized upon an addition of one of the (i) voice mode connection and (ii) the visual mode connection to the simultaneous voice mode and visual mode connection;

a state repository for maintaining information about the state of navigation in the application; and

an adaptive mode swapping logic for analyzing current conditions and determining whether to perform a mode swap based on said conditions.

23. A method of swapping between a first mode of communication and a second mode of communication, wherein an application and a user communicate through a telephone in said first and second modes of communication, the communication being in a first mode, wherein a first browser facilitates communication with application in the first mode, and a second browser facilitates communication with application in the second mode, the method comprising:

generating an instruction to perform a change of mode;

adding the second mode to the communication and dropping the first mode from the communication wherein at the instant of adding the second browser, the second browser is not yet synchronized to the state of the first browser;

synchronizing the second browser to a state of the first browser, further comprising the steps of;

maintaining state information of said first browser and said second browser;

receiving said maintained state information of first browser at the second browser; and

initializing the second browser to the state of the first browser;

whereby the first browser and the second browser are not required to be concurrently in the same updated state in all points in time during communication.