Multi-modal synchronization

• US 20030146932A1
• Filed: 04/25/2002
• Published: 08/07/2003
• Est. Priority Date: 02/07/2002
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP AG (SAP SE)
• Inventors: Weng, Jie, Raiyani, Samir, Vogler, Hartmut, Swan, Richard
• Granted Patent: US 7,203,907 B2
• Time in Patent Office: Days
• US Class Current: 345/748
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: H04M 2201/22 Synchronisation circuits
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

A first-modality gateway and a second-modality gateway are synchronized, with both gateways interfacing between a user and a server system. The synchronizing allows the user to use either of the first-modality gateway or the second-modality gateway at a given point in time to interface with specific information in the server system. A method includes accessing a communication sent from a first-modality gateway, and providing a synchronizing mechanism in response to accessing the communication. Another method includes receiving a request for a first-modality data from a first-modality entity, determining a second-modality data, and providing the second-modality data to a second-modality entity, where the second-modality data corresponds to the first-modality data. An article includes a first-modality interface, a second-modality interface, and a controller interface. The controller interface includes instructions that when executed by a machine result in interfacing the first-modality interface and the second-modality interface to a controller that provides synchronization.

Claims

1. A method comprising synchronizing a first-modality gateway and a second-modality gateway, both gateways interfacing between a user and a server system, the synchronizing allowing the user to use either of the first-modality gateway or the second-modality gateway at a given point in time to interface with specific information in the server system.

2. The method of claim 1 wherein synchronizing the first-modality gateway and the second-modality gateway comprises synchronizing a browser and a voice gateway, wherein the server system includes web pages.

3. The method of claim 1 further comprising synchronizing the first-modality gateway and a third-modality gateway, wherein the third-modality gateway interfaces between the user and the server system.

4. The method of claim 1 wherein synchronizing comprises:

sending a first-modality page from the server system to the first-modality gateway in response to receiving at the server system an input from the first-modality gateway;

determining a second-modality page that corresponds to the first-modality page; and

sending the second-modality page from the server system to the second-modality gateway.

5. The method of claim 4 wherein:

sending the first-modality page from the server system to the first-modality gateway comprises a web server sending a voice extensible markup language (“

VXML”

) page to a voice gateway in response to receiving at the web server the input from the voice gateway,

determining the second-modality page comprises the web server determining a hypertext markup language (“

HTML”

) page that corresponds to the VXML page,

sending the second-modality page comprises the web server sending the determined HTML page to a browser, and

the method further comprises the web server receiving a refresh request from the browser before the web server sends the determined HTML page.

6. The method of claim 4 wherein:

sending the first-modality page from the server system to the first-modality gateway comprises a web server sending a VXML page to a voice gateway in response to receiving at the web server the input from the voice gateway,

determining the second-modality page comprises the web server determining a HTML page that corresponds to the VXML page, and

sending the second-modality page to the second-modality gateway comprises the web server sending the determined HTML page to a browser by sending the determined HTML page to a publish/subscribe system with the result that the publish/subscribe system sends the determined HTML page to the browser.

7. The method of claim 4 wherein:

sending the first-modality page from the server system to the first-modality gateway comprises a web server sending a HTML page to a browser in response to receiving at the web server the input from the browser,

determining the second-modality page comprises the web server determining a VXML page that corresponds to the HTML page,

sending the second-modality page to the second-modality gateway comprises the web server sending the determined VXML page to a voice gateway by sending the VXML page to a publish/subscribe system with the result that the publish/subscribe system sends the VXML page to the voice gateway.

8. The method of claim 4 wherein:

sending the first-modality page from the server system to the first-modality gateway comprises a web server sending a HTML page to a browser in response to receiving at the web server the input from the browser,

determining the second-modality page comprises the web server determining a VXML page that corresponds to the HTML page,

sending the second-modality page to the second-modality gateway comprises the web server sending the determined VXML page to a voice gateway, and

the method further comprises the web server receiving a request from the voice gateway when the voice gateway has received no voice input for a specified amount of time, and the web server receives the request before the web server sends the determined VXML page to the voice gateway.

9. The method of claim 1 wherein synchronizing comprises:

receiving a request for a first-modality page from the first-modality gateway;

determining a second-modality page that corresponds to the first-modality page; and

requesting the second-modality page.

10. The method of claim 9 wherein:

receiving the request for the first-modality page from the first-modality gateway comprises a controller receiving the request for a HTML page from a browser, the controller being physically separate from the browser,

determining the second-modality page comprises the controller determining a VXML page that corresponds to the HTML page, and

requesting the second-modality page comprises the controller requesting the determined VXML page from a voice gateway, the controller being physically separate from the voice gateway and from the server system.

11. The method of claim 9 wherein:

receiving the request for the first-modality page from the first-modality gateway comprises a controller receiving the request for a VXML page from a voice gateway, the controller and the voice gateway residing on a mobile device,

determining the second-modality page comprises the controller determining a HTML page that corresponds to the VXML page, and

requesting the second-modality page comprises the controller requesting the determined HTML page from a web server.

12. The method of claim 1 wherein synchronizing comprises:

receiving a request for a first-modality page from the first-modality gateway;

determining a second-modality page that corresponds to the first-modality page; and

sending an identifier of the second-modality page to the second-modality gateway.

13. The method of claim 12 wherein:

receiving the request for the first-modality page from the first-modality gateway comprises a controller receiving the request for a HTML page from a browser, the controller and the browser residing on a mobile device,

determining a second-modality page comprises the controller determining a VXML page that corresponds to the HTML page, and

sending an identifier of the second-modality page to the second-modality gateway comprises the controller sending a uniform resource locator (“

URL”

) of the determined VXML page to a voice gateway, the voice gateway residing on the mobile device.

14. The method of claim 1 wherein synchronizing comprises:

receiving a request for a first-modality page from a first-modality interface;

accessing an identifier of a second-modality page, the second-modality page corresponding to the first-modality page; and

requesting the second-modality page.

15. The method of claim 14 wherein:

receiving the request for the first-modality page from the first-modality interface comprises a controller receiving the request for a VXML page from a voice interface, the controller being physically separate from the voice interface,

accessing the identifier of the second-modality page comprises the controller accessing a URL for a HTML page that corresponds to the VXML page, wherein the URL is supplied by a voice gateway that is physically separate from the controller, and

requesting the second-modality page comprises the controller requesting the corresponding HTML page for a browser from a web server, the controller being physically separate from the web server and the browser.

16. The method of claim 1 wherein synchronizing the first-modality gateway and the second-modality gateway comprises:

receiving from a first-modality interface a first item relating to first data; and

determining second data, the second data corresponding to the first data.

17. The method of claim 16 wherein:

receiving from the first-modality interface the first item relating to first data comprises a controller receiving from a voice interface a request for a VXML page, the first item comprising the request and the first data comprising the VXML page, the controller being physically separate from the voice interface,

determining second data, the second data corresponding to the first data, comprises the controller determining a HTML page that corresponds to the VXML page, and

the method further comprising the controller requesting the determined HTML page for a browser from a web server, the controller being physically separate from the browser and the web server.

18. The method of claim 16 further comprising providing the second data to the second-modality interface.

19. The method of claim 16 wherein:

receiving the first item comprises receiving a request for a first web page, the first web page being identified by a first uniform resource locator (“

URL”

) and the first data including the first URL; and

determining the second data comprises determining a second URL for a second web page, the second web page corresponding to the first web page, and the second data including the second URL.

20. The method of claim 1 wherein synchronizing the first-modality gateway and the second-modality gateway comprises:

receiving from the first-modality gateway a first item relating to a first data; and

determining a second data, the second data corresponding to the first data.

21. The method of claim 20 wherein a controller receives the first item, and the controller determines the second data.

22. The method of claim 20 further comprising providing the second data to the second-modality gateway.

23. The method of claim 22 further comprising waiting until after the second-modality gateway requests the second data before providing the second data to the second-modality gateway.

24. The method of claim 23 further comprising notifying the second-modality gateway to request the second data.

25. The method of claim 24 wherein:

notifying the second-modality gateway comprises a controller notifying the second-modality gateway through a publish/subscribe system, and

the controller provides the second data.

26. The method of claim 24 wherein:

notifying the second-modality gateway comprises a controller notifying the second-modality gateway by sending a refresh inquiry response, and

providing the second data comprises a controller providing the second data.

27. The method of claim 20 further comprising maintaining an identifier of the second data.

28. The method of claim 27 further comprising providing the second-modality gateway a pointer to the identifier of the second data.

29. The method of claim 20 further comprising requesting the second data from the server system.

30. The method of claim 29 wherein requesting the second data from the server system comprises a controller requesting the second data from a storage medium that is local with respect to the controller, the server system including the storage medium.

31. The method of claim 29 wherein requesting the second data from the server system comprises a controller requesting the second data from a storage medium that is remote with respect to the controller, the server system including the storage medium.

32. The method of claim 20 further comprising providing an identifier of the second data to the second-modality gateway.

33. The method of claim 20 wherein determining the second data comprises receiving an identifier of the second data from the first-modality gateway.

34. A method comprising:

accessing a communication sent from a first-modality gateway; and

providing a synchronizing mechanism in response to accessing the communication, the synchronizing mechanism being capable of use in synchronizing the first-modality gateway and a second-modality gateway, both gateways interfacing between a user and a server system.

35. The method of claim 34 wherein:

accessing a communication comprises receiving a request for a first item, and

providing a synchronizing mechanism comprises

determining a second item that corresponds to the first item, and

sending the second item to the second-modality gateway.

36. The method of claim 34 wherein:

accessing a communication comprises receiving a request for a first item, and

providing a synchronizing mechanism comprises

determining a second item that corresponds to the first item,

determining an identifier of the second item, and

sending the identifier to the second-modality gateway.

37. The method of claim 34 wherein:

accessing a communication comprises receiving an item, and

providing a synchronizing mechanism comprises

determining the identity of the second-modality gateway, and

sending the item to the second-modality gateway.

38. The method of claim 37 wherein receiving an item comprises receiving a text entry entered by a user.

39. A method comprising:

receiving a request for a first-modality data from a first-modality entity;

determining a second-modality data, the second-modality data corresponding to the first-modality data; and

providing the second-modality data to a second-modality entity.

40. The method of claim 39 wherein the first-modality entity comprises a first-modality gateway, and the second-modality entity comprises a second-modality gateway.

41. An article comprising a storage medium having stored thereon instructions that when executed by a machine result in synchronizing a first-modality gateway and a second-modality gateway, both gateways being adapted to interface between a user and a server system, the synchronizing allowing the user to use either of the first-modality gateway or the second-modality gateway at a given point in time to interface with specific information in the server system.

42. The article of claim 41 further comprising a first-modality interface adapted to be coupled to the first-modality gateway, and a second-modality interface adapted to be coupled to the second-modality gateway.

43. An article comprising:

a first-modality interface;

a second-modality interface; and

a controller interface, wherein the controller interface includes instructions that when executed by a machine result in interfacing the first-modality interface and the second-modality interface to a controller that provides synchronization of the first-modality interface and the second-modality interface.

44. The article of claim 43 wherein the first-modality interface comprises a browser interface.

45. A system comprising one or more components implemented in hardware, firmware, or software, the one or more components being adapted to perform at least the following:

receiving a request for a first-modality page from a first-modality gateway;

determining a second-modality page that corresponds to the first-modality page; and

requesting the second-modality page.

46. The system of claim 45 wherein the first-modality gateway comprises a browser, and the second modality gateway comprises a voice gateway.