Multi-modal communication using a session specific proxy server

• US 20030140113A1
• Filed: 12/28/2001
• Published: 07/24/2003
• Est. Priority Date: 12/28/2001
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Inventors: Balasuriya, Senaka
• Granted Patent: US 8,799,464 B2
• Time in Patent Office: Days
• US Class Current: 709/217
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: H04L 65/1045 Proxies, e.g. for session initiation protocol [SIP]
• CPC Class Codes: H04L 65/1046 Call controllers; Call servers
• CPC Class Codes: H04L 67/141 Setup of application sessions admission control or resource allocation in data switching networks H04L47/70

Abstract

A method and apparatus for multi-modal communication includes a controller (236) operably coupled to at least one multi-modal session proxy server (226). On a per multi-modal session basis, the controller (236) provides the multi-modal session proxy server (226) with a multi-modal proxy identifier (138). The multi-modal proxy identifier (138) is then provided to at least one browser with a per session multi-modal proxy evaluator (220) having a browser proxy identifier (140) wherein the browser proxy identifier (140) is evaluated in view of the multi-modal proxy identifier (138). The multi-modal session proxy server (226) then receives an information request (231) from the browser with per session multi-modal proxy evaluator (220) wherein the requested information is fetched from a content server (240). When the requested information is retrieved, a multi-modal synchronization coordinator (122) notifies the other browser with per session multi-modal proxy evaluator (232), via a multi-modal synchronization interface (234).

Claims

1. An apparatus for multi-modal communication comprising:

a controller; and

at least one multi-modal session proxy server having a proxy address, wherein the controller determines, on a per session basis, a multi-modal proxy identifier representing the proxy address of the multi-modal session proxy server.

2. The apparatus of claim 1 further comprising:

at least one browser having a per session multi-modal proxy evaluator and a browser proxy identifier, wherein the browser is operably coupled to the controller and the at least one multi-modal session proxy server such that the browser receives the multi-modal proxy identifier and the browser proxy identifier is evaluated by the multi-modal proxy evaluator, on a per session basis, in response to the multi-modal proxy identifier.

3. The apparatus of claim 1 further comprising:

at least one voice browser having a voice browser per session multi-modal proxy evaluator and a voice browser proxy identifier, wherein the voice browser is operably coupled to the controller and the at least one multi-modal session proxy server such that the voice browser receives the multi-modal proxy identifier and the voice browser proxy identifier is evaluated by the voice browser per session multi-modal proxy evaluator, on a per session basis, in response to the multi-modal proxy identifier; and

at least one graphical browser having a graphical browser per session multi-modal proxy evaluator and a graphical browser proxy identifier, wherein the graphical browser is operably coupled to the controller and the at least one multi-modal session proxy server such that the graphical browser receives the multi-modal proxy identifier and the graphical browser proxy identifier is evaluated by the graphical browser per session multi-modal proxy evaluator, on a per session basis, in response to the multi-modal proxy identifier.

4. The apparatus of claim 3 further comprising:

at least one graphical browser multi-modal synchronization interface operably coupled to the graphical browser;

at least one voice browser multi-modal synchronization interface operably coupled to at least one the voice browser; and

at least one multi-modal synchronization coordinator operably coupled to the graphical browser multi-modal synchronization interface, the voice browser multi-modal synchronization interface and the multi-modal session proxy, wherein multi-modal session proxy server allows the multi-modal synchronization coordinator to synchronize the at least one graphical browser and the at least one voice browser.

5. The apparatus of claim 4 further comprising:

at least one information request provided by at least one of the at least graphical browser and the at least one voice browser to the multi-modal session proxy server whereby the multi-modal session proxy server fetches requested information from a content server; and

wherein if the requested information is provided to the at least one voice browser, the at least one graphical browser is updated via the at least one graphical browser multi-modal synchronization interface through the multi-modal synchronization coordinator and if the requested information is provided to the at least one graphical browser, the at least one voice browser is updated via the voice browser multi-modal synchronization interface through the multi-modal synchronization coordinator.

6. The apparatus of claim 1 wherein the controller further comprises at least one load balancer whereupon the controller determines the multi-modal proxy identifier in response to the at least one load balancer.

7. A system for multi-modal communication comprising:

a terminal that includes at least;

at least one graphical browser with a graphical browser per session multi-modal proxy evaluator, wherein the at least one graphical browser dynamically evaluates, on a per session basis, a graphical browser proxy identifier; and

at least one graphical browser multi-modal synchronization interface operably coupled to the at least one graphical browser.

8. The system of claim 7 wherein the terminal further comprises:

an audio subsystem comprising a microphone for receiving an audio input and a speaker for providing an audio output;

a terminal voice transfer interface operably coupled to the audio subsystem; and

a terminal session control for initiating a wireless terminal session.

9. The system of claim 8 further comprising:

a network element including;

at least one voice browser with a voice browser per session multi-modal proxy evaluator, wherein the at least one voice browser dynamically evaluates, on a per session basis a voice browser proxy identifier; and

at least one voice browser multi-modal synchronization interface operably coupled to the at least one voice browser.

10. The system of claim 9 wherein the network element includes:

a network session control for receiving initialization commands from the terminal session control;

a network voice transfer interface for transmitting and receiving encoded audio information from the terminal voice transfer interface; and

a speech engine operably coupled to the network voice transfer interface, wherein when a voice command is provided from the terminal voice transfer interface, the speech engine decodes the voice command for the voice browser.

11. The system of claim 10 wherein the network element further includes:

a controller; and

at least one multi-modal session proxy server having a proxy address operably coupled to the at least one voice browser and the at least one graphical browser, wherein the controller determines, on a per session basis, a multi-modal proxy identifier representing the proxy address of the multi-modal session proxy server and the multi-modal proxy identifier is provided to the at least one voice browser and the at least one graphical browser.

12. The system of claim 11 further comprising:

a multi-modal synchronization coordinator;

at least one information request provided by at least one of the at least one graphical browser and the at least one voice browser to the multi-modal session proxy server whereby the multi-modal session proxy server fetches requested information; and

wherein if the requested information is provided to the at least one voice browser, the at least one graphical browser is updated via the at least one graphical browser multi-modal synchronization interface through the multi-modal synchronization coordinator and if the requested information is provided to the at least one graphical browser, the at least one voice browser is updated via the at least one voice browser multi-modal synchronization interface through the multi-modal synchronization coordinator.

13. The system of claim 11 further comprising:

at least one content server operably coupled to the multi-modal session proxy server of the network element whereupon when an information request is provided to the multi-modal session proxy server, the multi-modal session proxy server retrieves the requested information from the at least one content server.

14. The system of claim 11 wherein the controller of the network element further comprises:

a load balancer operably coupled to the controller wherein the load balancer provides an indication of the available bandwidth of the at least one multi-modal proxy server to the controller such that the controller can determine, on a per session basis, the multi-modal session proxy identifier.

15. An apparatus for multi-modal communication comprising:

a first browser with a per session multi-modal proxy evaluator and a first browser proxy identifier, the first browser operably coupled to a first browser multi-modal synchronization interface;

at least one second browser with a per session multi-modal proxy evaluator and a second browser proxy identifier, the at least one second browser operably coupled to a second browser multi-modal synchronization interface;

a multi-modal synchronization coordinator operably coupled to the first browser multi-modal synchronization interface and the second browser multi-modal synchronization interface, whereupon the multi-modal synchronization coordinator synchronizes the first browser and the at least one second browser during a multi-modal session; and

at least one multi-modal session proxy server having a proxy address, the multi-modal session proxy server operably coupled to the first browser, the at least one second browser and the multi-modal synchronization coordinator, wherein the first browser dynamically updates the first browser proxy identifier on a per session basis and the second browser dynamically updates the second browser proxy identifier on a per session basis.

16. The apparatus of claim 15 further comprising:

a controller operably coupled to the multi-modal session proxy server, wherein the controller determines, on a per session basis, a multi-modal proxy identifier representing the proxy address and provides the multi-modal proxy identifier to the first browser and the at least one second browser.

17. The apparatus of claim 16 further comprising:

at least one information request provided by at least one of the first browser and the at least one second browser to the multi-modal session proxy server whereby the multi-modal session proxy fetches requested information from a content server; and

wherein if the requested information is provided to the at least one second browser, the first browser is updated via the first browser multi-modal synchronization interface through the multi-modal synchronization coordinator and if the requested information is provided to the first browser, the at least one second browser is updated via the second browser multi-modal synchronization interface through the multi-modal synchronization coordinator.

18. The apparatus of claim 17 wherein the first browser is a graphical browser and the at least one second browser is a voice browser.

19. A method for multi-modal communication comprising:

receiving a multi-modal proxy identifier, on a per session basis, for a browser;

evaluating, on a per session basis, a browser proxy identifier in response to receiving the multi-modal proxy identifier; and

sending an information request via a multi-modal session proxy server identified by the multi-modal proxy identifier.

20. The method of claim 19 further comprising:

fetching requested information from at least one content server; and

providing the requested information to the browser.

21. The method of claim 20 further comprising:

prior to sending an information request, storing an updated browser proxy identifier in a memory location.

22. A method for multi-modal communication comprising:

receiving a multi-modal proxy identifier, on a per session basis, for a graphical browser;

evaluating, on a per session basis, a graphical browser proxy identifier in response to receiving the multi-modal proxy identifier

receiving the multi-modal proxy identifier, on a per session basis, for a voice browser;

evaluating, on a per session basis, a voice browser proxy identifier in response to receiving the multi-modal proxy identifier

sending an information request from at least one of the graphical browser or voice browser via a multi-modal session proxy server identified by the multi-modal proxy identifier.

23. The method of claim 22 further comprising:

prior to receiving a multi-modal proxy identifier, initiating a multi-modal session between a terminal and a multi-modal network element and determining the multi-modal session proxy server on a per session basis.

24. The method of claim 23 wherein the graphical browser is disposed on the terminal and the voice browser is disposed on the network element.

25. The method of claim 24, prior to sending an information request, further comprising:

storing an updated graphical browser proxy identifier in a graphical browser memory location; and

storing an updated voice browser proxy identifier in a voice browser memory location.

26. A method for multi-modal communication comprising:

determining a multi-modal session proxy server, on a per session basis; and

providing, on a per session basis, a multi-modal proxy identifier to a browser.

27. The method of claim 26, the step of determining a multi-modal session proxy server, on a per session basis, further comprising:

accessing a load balancer, wherein the load balancer is operably coupled to a controller; and

determining the multi-modal session proxy server in response to the load balancer.

28. The method of claim 26 further comprising:

prior to determining a multi-modal session proxy server, on a per session basis, initiating a multi-modal session between a terminal and a multi-modal network element.

29. The method of claim 28 further comprising:

evaluating, on a per session basis, a browser proxy identifier in response to receiving the multi-modal proxy identifier; and

receiving an information request from the browser to the multi-modal session proxy server identified by the multi-modal proxy identifier.

30. The method of claim 28 further comprising:

fetching requested information from a content server; and

providing the requested information to the browser.

31. A method for multi-modal communication comprising:

generating a multi-modal proxy identifier and a multi-modal session identifier;

providing the multi-modal proxy identifier and the multi-modal session identifier to a first multi-modal browser and at least one second multi-modal browser;

sending an information fetch request for requested information from the first multi-modal browser, the information fetch request having the multi-modal session identifier, to a multi-modal session proxy server identified by the multi-modal proxy identifier; and

fetching the requested information from a content server and caching the requested information in the multi-modal proxy server, referencing the requested information by the multi-modal session identifier.

32. The method of claim 31 further comprising:

providing the requested information to the first multi-modal browser;

notifying a multi-modal synchronization coordinator of the successful fetch request; and

notifying the at least one second multi-modal browser of the first multi-modal browser information fetch request.

33. The method of claim 32 further comprising:

sending a second information fetch request from the at least one second multi-modal browser to the multi-modal session proxy identified by the multi-modal proxy identifier, wherein the fetch request has the multi-modal session identifier.

34. The method of claim 33 further comprising:

retrieving the requested information from the multi-modal session proxy server based on the multi-modal session identifier; and

providing the requested information to the at least one second multi-modal browser.