Multi-modal web interaction over wireless network

• US 8,566,103 B2
• Filed: 12/22/2010
• Issued: 10/22/2013
• Est. Priority Date: 11/13/2002
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: He, Liang
• Primary Examiner(s): GUERRA-ERAZO, EDGAR X
• Application Number:
  US12/976,320
• Publication Number:
  US 20110202342A1
• Time in Patent Office: 1,035 Days
• Field of Search: 704/200, 704/270, 704/270.1, 704/272, 704/275, 704/251, 704/231, 704/235, 704/246, 704/277, 715/853, 715/738
• US Class Current: 704/270.1
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML
• CPC Class Codes: H04W 80/00 Wireless network protocols or protocol adaptations to wireless operation

Abstract

A system, apparatus, and method is disclosed for receiving user input at a client device, interpreting the user input to identify a selection of at least one of a plurality of web interaction modes, producing a corresponding client request based in part on the user input and the web interaction mode; and sending the client request to a server via a network.

Claims

1. A method comprising:

receiving, at a server, a session message from a client device via a network, the session message requesting establishment of a session with the server and comprising (i) a voice type requested by the client device and (ii) a user language requested by the client device;

determining, at the server, if the requested voice type and the requested user language are supported by the server;

sending, by the server, a ready message to the client device via the network in response to determining that the requested voice type and the requested user language are supported by the server;

receiving, at the server, a client request from the client device via the network subsequent to sending the ready message to the client device, the client request including a focused group of hyperlinks and speech data;

interpreting the client request to identify a selection of at least one of a plurality of web interaction modes, at least one web interaction mode being a speech interaction mode; and

building a correct grammar for speech recognition based on the speech data and the focused group of hyperlinks, performing speech recognition, and performing specific tasks according to the result of the speech recognition.

3. The method of claim 1, further comprising:

dividing, by the server, the web page into a plurality of multi-modal markup language cards, each multi-modal markup language card comprises a plurality of display elements; and

transmitting, by the server, one or more of the plurality of multi-modal markup language cards to the client device.

4. The method of claim 1, further comprising:

determining a recognition output score for the result of the speech recognition; and

determining whether the recognition output score meets a threshold percentage of acceptable recognition accuracy.

5. The method of claim 1, further comprising:

determining, by the client device, a period of time during which the client device is inactive; and

un-focusing, by the client device, the focused group of hyperlinks in response to determining that the period of time during which the client device is inactive exceeds a reference timeout period of time.

6. The method of claim 1, further comprising highlighting, by the client device, the focused group of hyperlinks.

7. The method of claim 1, wherein the speech data comprises a speech element bound to each hyperlink of the focused group of hyperlinks.

8. The method of claim 1, wherein the focused group of hyperlinks comprises a focused form.

16. The method of claim 1, further comprising sending, by the server, an error message to the client device in response to determining that the requested voice type and the requested user language are not supported by the server.

17. The method of claim 1, wherein the requested user language comprising a language spoken by a user of the client device.

18. The method of claim 1, further comprising receiving, at the server, requested transmission parameters from the client device subsequent to sending the ready message to the client device, the requested transmission parameters comprising a quality of service (QoS) level and a bandwidth requested by the client device.

19. The method of claim 18, further comprising sending, by the server to the client device, the QoS level and the bandwidth to be used by the client device in response to receiving the requested transmission parameters from the client device,

wherein receiving, at the server, a client request from the client device via the network subsequent to sending the ready message to the client device comprising receiving, at the server, a client request from the client device via the network subsequent to sending, by the server to the client device, the QoS level and the bandwidth to be used by the client device.

20. The method of claim 1, further comprising sending, by the server to the client device, a quality of service (QoS) level and a bandwidth to be used by the client device in response to a change in network status.

2. A non-transitory machine-readable medium having stored thereon data representing instructions which, when executed by a machine, cause the machine to perform operations, comprising:

receiving a session message from a client device via a network, the session message requesting establishment of a session and comprising (i) a voice type requested by the client device and (ii) a user language requested by the client device;

determining if the requested voice type and the requested user language are supported;

sending a ready message to the client device via the network in response to determining that the requested voice type and the requested user language are supported;

receiving a client request from the client device via the network subsequent to sending the ready message to the client device, the client request including a focused group of hyperlinks and speech data;

interpreting the client request to identify a selection of at least one of a plurality of web interaction modes, at least one web interaction mode being a speech interaction mode; and

building a correct grammar for speech recognition based on the speech data and the focused group of hyperlinks, performing speech recognition, and performing specific tasks according to the result of the speech recognition.

9. The non-transitory machine-readable medium of claim 2, wherein the instructions further cause the machine to perform operations, comprising:

dividing the web page into a plurality of multi-modal markup language cards, each multi-modal markup language card comprises a plurality of display elements; and

transmitting one or more of the plurality of multi-modal markup language cards to the client device.

10. The non-transitory machine-readable medium of claim 2, wherein the instructions further cause the machine to perform operations, comprising:

determining a recognition output score for the result of the speech recognition; and

determining whether the recognition output score meets a threshold percentage of acceptable recognition accuracy.

11. The non-transitory machine-readable medium of claim 2, wherein the speech data comprises a speech element bound to each hyperlink of the focused group of hyperlinks.

12. The non-transitory machine-readable medium of claim 2, wherein the instructions further cause the machine to perform operations, comprising sending an error message to the client device in response to determining that the requested voice type and the requested user language are not supported.

13. The non-transitory machine-readable medium of claim 2, wherein the instructions further cause the machine to perform operations, comprising receiving requested transmission parameters from the client device subsequent to sending the ready message to the client device, the requested transmission parameters comprising a quality of service (QoS) level and a bandwidth requested by the client device.

14. The non-transitory machine-readable medium of claim 13, wherein the instructions further cause the machine to perform operations, comprising sending, to the client device, the QoS level and the bandwidth to be used by the client device in response to receiving the requested transmission parameters from the client device,

wherein receiving a client request from the client device via the network subsequent to sending the ready message to the client device comprising receiving a client request from the client device via the network subsequent to sending, to the client device, the QoS level and the bandwidth to be used by the client device.

15. The non-transitory machine-readable medium of claim 2, wherein the instructions further cause the machine to perform operations, comprising sending, to the client device, a quality of service (QoS) level and a bandwidth to be used by the client device in response to a change in network status.