Sequential multimodal input

• US 7,363,027 B2
• Filed: 11/11/2003
• Issued: 04/22/2008
• Est. Priority Date: 11/11/2003
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Kuansan, Hon, Hsiao-Wuen
• Primary Examiner(s): Perez-Gutierrez; Rafael
• Assistant Examiner(s): Ajibade-Akonai; Olumide
• Application Number:
  US10/705,019
• Publication Number:
  US 20050101300A1
• Time in Patent Office: 1,624 Days
• Field of Search: 455/414.1, 455/414.2, 455/414.3, 455/414.4, 455/563, 704/270.1, 704/235, 704/275, 709/202
• US Class Current: 455/414.1
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

A method of interacting with a client/server architecture with a 2.5G mobile phone having a data channel for transmitting data and a voice channel for transmitting speech. The method includes receiving a web page from a web server pursuant to an application through the data channel and rendering the web page on the 2.5G phone, where rendering comprises processing the web page to be responsive speech input. Speech is received from the user corresponding to at least one data field on the web page. A call is established from the 2.5G phone to a telephony server over the voice channel. The telephony server is remote from the 2.5G phone and adapted to process speech. A speech-enabled web page is obtained from the web server corresponding to the web page provided to the 2.5G phone. Speech is transmitted from the 2.5G phone to the telephony server. The speech is processed in accordance with the speech-enabled web page to obtain textual data in accordance with the speech. The textual data is transmitted to the web server. A new web page is obtained on the 2.5G phone through the data channel and rendered having the textual data.

Claims

1. A method of interacting with a client/server architecture with a 2.5G mobile phone, the 2.5G phone having a data channel for transmitting data and a voice channel for transmitting speech, the method comprising:

receiving a web page from a web server pursuant to an application through the data channel and rendering the web page on the 2.5G phone, wherein the web page comprises at least one data field, and wherein rendering comprises processing the web page to be responsive to speech input;

receiving speech from the user corresponding to the at least one data field on the web page;

establishing a call from the 2.5G phone to a telephony server over the voice channel, the telephony server being remote from the 2.5G phone and adapted to process speech;

obtaining a speech-enabled web page from the web server corresponding to the web page provided to the 2.5G phone;

transmitting speech from the 2.5G phone to the telephony server;

processing the speech in accordance with the speech-enabled web page to obtain textual data in accordance with the speech, wherein the textual data corresponds with the at least one data field;

transmitting the textual data to the web server; and

obtaining a new web page on the 2.5G phone through the data channel and rendering the new web page having the textual data entered at the at least one data field.

2. The method of claim 1 wherein processing the speech includes transmitting data indicative of the received speech to a speech server remote from the telephony server, the speech server processing the data indicative of the received speech to obtain the textual data, and wherein transmitting the textual data to the web server comprises the speech server transmitting the textual data.

3. The method of claim 1 wherein establishing a call from the 2.5G phone to a telephony server over the voice channel includes obtaining an identifier associated with the 2.5G phone.

4. The method of claim 3 wherein obtaining a speech-enabled web page from the web server corresponding to the web page provided to the 2.5G phone includes using the identifier associated with the 2.5G phone.

5. The method of claim 4 wherein obtaining an identifier includes identifying a phone number associated with the 2.5G phone.

6. The method of claim 1 and further comprising, prior to obtaining the new web page from the web server, detecting a completion of speech input by the telephony server, disconnecting the voice channel, and requesting a new web page from the web server.

7. The method of claim 6 and further comprising transmitting speech data corresponding to at least one other data field on the rendered web page to the web server.

8. The method of claim 7 and further comprising combining at the web server said non-speech data for said at least one other field with the textual data to form the new web page.

9. The method of claim 8 and further comprising disconnecting the voice channel prior to the telephony server obtaining the new web page.

10. The method of claim 2, wherein the remote speech server is adapted to be provided by a third party different than a provider of the application on the web server.

14. The method of claim 1 and further comprising, prior to obtaining the new web page from the web server, detecting a completion of speech input by the telephony server, disconnecting the voice channel, and requesting the new web page from the web server.

15. The method of claim 14 and further comprising transmitting non-speech data corresponding to at least one other data field on the rendered web page for said at least one other field to the web server.

11. A method of interacting with a client/server architecture with a 2.5G mobile phone, the 2.5G phone having a data channel for transmitting data and a voice channel for transmitting speech, the method comprising:

receiving a web page from a web server pursuant to an application through the data channel and rendering the web page on the 2.5G phone, wherein rendering comprises processing the web page to be responsive speech input, wherein the web age comprises at least one data field;

receiving speech from the user corresponding to the at least one data field on the web page;

establishing a call from the 2.5G phone to a telephony server over the voice channel, the telephony server being remote from the 2.5G phone and adapted to process speech;

transmitting speech from the 2.5G phone to the telephony server; and

obtaining an updated web page on the 2.5G phone through the data channel and rendering the updated web page having the corresponding at least one data field updated in accordance with the speech.

12. The method of claim 11 wherein establishing a call from the 2.5G phone to a telephony server over the voice channel includes sending an identifier associated with the 2.5G phone.

13. The method of claim 12 wherein sending an identifier includes sending a phone number associated with the 2.5G phone.

16. A method of interacting with a client/server architecture with a 2.5G mobile phone, the 2.5G phone having a data channel for transmitting data and a voice channel for transmitting speech, the method comprising:

transmitting a web page from a web server pursuant to an application through the data channel suitable for rendering the web page on the 2.5G phone, wherein the web page comprises at least one data field, and wherein rendering comprises processing the web page to be responsive to speech input;

transmitting a speech-enabled web page to a telephony server from the web server corresponding to the web page provided to the 2.5G phone;

receiving textual data based on speech provided to the telephony server, the textual data being indicative of recognized speech for the at least one data field; and

transmitting an updated web page to the 2.5G phone through the data channel with the textual data input at the corresponding at least one data field.

17. The method of claim 16 wherein receiving textual data includes receiving textual data from a speech server remote from the telephony server, wherein the speech server processed the speech provided to the telephony server.