Sequential multimodal input

• US 7,158,779 B2
• Filed: 11/11/2003
• Issued: 01/02/2007
• Est. Priority Date: 11/11/2003
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Kuansan, Hon, Hsiao-Wuen
• Primary Examiner(s): Gelin; Jean
• Assistant Examiner(s): Ajibade-Akonai; Olumide
• Application Number:
  US10/705,155
• Publication Number:
  US 20050101355A1
• Time in Patent Office: 1,148 Days
• Field of Search: 455/414.1, 455/414.2, 455/414.3, 455/414.4, 455/466, 455/563, 370/352, 370/466, 370/401, 370/338, 370/353, 370/354, 370/355, 370/356, 704/270, 704/270.1, 704/275, 704/235, 709/217, 709/218, 709/219, 709/227, 709/228, 709/229, 379/88.17, 379/88.18
• US Class Current: 455/414.1
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: H04M 1/72445 for supporting Internet browser applications
• CPC Class Codes: H04M 2201/38 Displays
• CPC Class Codes: H04M 2207/18 wireless networks
• CPC Class Codes: H04M 2250/22 including a touch pad, a touch sensor or a touch detector digitizers for touch screens G06F3/041
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML
• CPC Class Codes: H04M 7/0027 Collaboration services where a computer is used for data transfer and the telephone is used for telephonic communication

Abstract

A method of interacting with a client/server architecture with a 2G mobile phone is provided. The 2G phone includes a data channel for transmitting data and a voice channel for transmitting speech. The method includes receiving a web page from a web server pursuant to an application through the data channel and rendering the web page on the 2G phone. Speech is received from the user corresponding to at least one data field on the web page. A call is established from the 2G phone to a telephony server over the voice channel. The telephony server is remote from the 2G phone and is adapted to process speech. The telephony server obtains a speech-enabled web page from the web server corresponding to the web page provided to the 2G phone. Speech is transmitted from the 2G phone to the telephony server. The speech is processed in accordance with the speech-enabled web page to obtain textual data. The textual data is transmitted to the web server. The 2G phone obtains a new web page through the data channel and renders the new web page having the textual data.

Claims

1. A method of interacting with a client/server architecture with a 2G mobile phone, the 2G phone having a data channel for transmitting data and a voice channel for transmitting speech, the method comprising:

receiving a web page from a web server pursuant to an application through the data channel and rendering the web page on the 2G phone, wherein the web page comprises at least one data field;

receiving speech from the user corresponding to the at least one data field on the web page;

establishing a call from the 2G phone to a telephony server over the voice channel, the telephony server being remote from the 2G phone and adapted to process speech;

obtaining a speech-enabled web page by the telephony server from the web server corresponding to the web page provided to the 2G phone;

transmitting the speech from the 2G phone to the telephony server;

processing the speech in accordance with the speech-enabled web page by the telephony server to obtain textual data in accordance with the speech, the textual data corresponding to the at least one data field,;

transmitting the textual data from the telephony server to the web server; and

obtaining a new web page on the 2G phone through the data channel and rendering the new web page having the textual data input at the corresponding at least one data field.

2. The method of claim 1 wherein processing the speech includes transmitting data indicative of the received speech to a speech server remote from the telephony server, the speech server processing the data indicative of the received speech to obtain the textual data, and wherein transmitting the textual data to the web server comprises the speech server transmitting the textual data.

3. The method of claim 1 wherein establishing a call from the 2G phone to a telephony server over the voice channel includes obtaining an identifier associated with the 2G phone.

4. The method of claim 3 wherein obtaining a speech-enabled web page from the web server corresponding to the web page provided to the 2G phone includes using the identifier associated with the 2G phone.

5. The method of claim 4 wherein the obtaining an identifier includes identifying a phone number associated with the 2G phone.

6. The method of claim 1 and further comprising, prior to obtaining the new web page from the web server, transmitting a message to the 2G phone that a new page is available from the web server.

7. The method of claim 6 wherein transmitting the message comprises transmitting a SMS message.

8. The method of claim 6 wherein transmitting the message comprises transmitting information pertaining to an address for the new web page.

9. The method of claim 8 wherein transmitting the message comprises transmitting a URL link.

10. The method of claim 9 wherein transmitting the message comprises transmitting a SMS message.

11. The method of claim 6 and further comprising disconnecting the voice channel prior to the telephony server obtaining the new web page.

12. The method of claim 2, wherein the remote speech server is adapted to be provided by a third party different than a provider of the application on the web server.

13. A method of interacting with a client/server architecture with a 2G mobile phone, the 2G phone having a data channel for transmitting data and a voice channel for transmitting speech, the method comprising:

receiving a web page from a web server pursuant to an application through the data channel and rendering the web page on the 2G phone, wherein the web page comprises at least one data field;

receiving speech from the user, the speech corresponding to the at least one data field on the web page;

establishing a call from the 2G phone to a telephony server over the voice channel, the telephony server being remote from the 2G phone and adapted to process speech;

transmitting the speech from the 2G phone to the telephony server; and

obtaining an updated web page on the 2G phone through the data channel and rendering the updated web page having the corresponding at least one data field updated in accordance with the speech.

14. The method of claim 13 wherein establishing a call from the 2G phone to a telephony server over the voice channel includes sending an identifier associated with the 2G phone.

15. The method of claim 14 wherein sending an identifier includes identifying a phone number associated with the 2G phone.

16. The method of claim 13 and further comprising, prior to obtaining the new web page from the web server, receiving a message that a new page is available from the web server.

17. The method of claim 16 wherein receiving the message comprises receiving a SMS message.

18. The method of claim 16 wherein receiving the message comprises receiving information pertaining to an address for the new web page.

19. The method of claim 18 wherein receiving the message comprises receiving a URL link.

20. A method of interacting with a client/server architecture with a 2G mobile phone, the 2G phone having a data channel for transmitting data and a voice channel for transmitting speech, the method comprising:

transmitting a web page from a web server pursuant to an application through the data channel suitable for rendering on a 2G phone, wherein the web page comprises at least one data field;

transmitting a speech-enabled web page to a telephony server from the web server corresponding to the web page provided to the 2G phone;

receiving textual data from the telephony server indicative of recognized speech for the at least one data field; and

transmitting an updated web page to the 2G phone through the data channel with the textual data input at the corresponding at least one data field.