Establishing a preferred mode of interaction between a user and a multimodal application

• US 9,292,183 B2
• Filed: 06/20/2013
• Issued: 03/22/2016
• Est. Priority Date: 09/11/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Cross, Charles W. Jr., Pike, Hilary A.
• Primary Examiner(s): VO, HUYEN X
• Application Number:
  US13/922,969
• Publication Number:
  US 20130283172A1
• Time in Patent Office: 1,006 Days
• Field of Search: 704 1- 10, 704/251, 704/255, 704/257, 704/270, 704/270.1, 704/275, 713/189, 710/18
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: H04M 3/42153 by subscriber
• CPC Class Codes: H04M 3/4936 Speech interaction details speech recognition per se G10L15/00
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML
• CPC Class Codes: H04N 21/4126 The peripheral being portable, e.g. PDAs or mobile phones

Abstract

Establishing a preferred mode of interaction between a user and a multimodal application, including evaluating, by a multimodal application operating on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, user modal preference, and dynamically configuring multimodal content of the multimodal application in dependence upon the evaluation of user modal preference.

Claims

1. An apparatus, comprising:

at least one computer processor;

at least one communications interface configured to transmit information to a web server over a network; and

at least one computer-readable medium configured to store a plurality of computer program instructions that, when executed by the at least one computer processor, perform a method comprising;

receiving from the web server, by the apparatus, a web page for a multimodal application wherein the web page includes a plurality of input fields;

presenting the web page on a display of the apparatus;

monitoring user input to determine a mode of interaction used by a user when interacting, via the apparatus, with at least one input field of the web page;

storing information indicating the determined mode of interaction used by the user when interacting with the at least one input field;

evaluating, by the at least one computer processor, a user modal preference based, at least in part, on the stored information indicating the determined mode of interaction used by the user when interacting with the at least one input field; and

sending an indication of the user modal preference from the apparatus to the web server via the at least one communications interface.

2. The apparatus of claim 1, wherein the apparatus comprises a mobile device.

3. The apparatus of claim 1, wherein evaluating the user modal preference further comprises determining that the user modal preference is the user modal preference indicated in the stored information.

4. The apparatus of claim 1, wherein the method further comprises:

receiving, from the web server, multimodal content for the multimodal application configured based, at least in part, on the user modal preference; and

presenting the multimodal content to the user via the apparatus.

5. The apparatus of claim 1, wherein the method further comprises:

receiving a configuration value indicating the user modal preference; and

storing the configuration value;

wherein evaluating the user modal preference comprises retrieving the stored configuration value.

6. The apparatus of claim 1, wherein storing information indicating the determined mode of interaction used by the user when interacting with the at least one input field comprises sending the information indicating the determined mode of interaction to a storage device connected to the apparatus via at least one network.

7. A method of establishing a preferred mode of interaction between a user and a multimodal application, the method comprising:

receiving from a web server, by an apparatus, a web page for a multimodal application, wherein the web page includes a plurality of input fields;

presenting the web page on a display of the apparatus;

monitoring user input to determine a mode of interaction used by a user when interacting, via the apparatus, with at least one input field of the web page;

storing information indicating the determined mode of interaction used by the user when interacting with the at least one input field;

evaluating, by at least one computer processor, a user modal preference based, at least in part, on the stored information indicating the determined mode of interaction used by the user when interacting with the at least one input field; and

sending an indication of the user modal preference from the apparatus to the web server via at least one communications interface.

8. The method of claim 7, further comprising presenting the multimodal application on a mobile device.

9. The method of claim 7, wherein evaluating the user modal preference further comprises determining that the user modal preference is the user modal preference indicated in the stored information.

10. The method of claim 7, further comprising:

receiving, from the web server, multimodal content for the multimodal application configured based, at least in part, on the user modal preference; and

presenting the multimodal content to the user via the apparatus.

11. The method of claim 7, further comprising:

receiving a configuration value indicating the user modal preference; and

storing the configuration value;

wherein evaluating the user modal preference comprises retrieving the stored configuration value.

12. The apparatus of claim 7, wherein storing information indicating the determined mode of interaction used by the user when interacting with the at least one input field comprises sending the information indicating the determined mode of interaction to a storage device connected to the apparatus via at least one network.

13. A non-transitory computer-readable medium encoded with computer program instructions that, when executed by a computer processor of an apparatus, perform a method of establishing a preferred mode of interaction between a user and a multimodal application, the method comprising:

receiving from a web server, by the apparatus, a web page for a multimodal application, wherein the web page includes a plurality of input fields;

presenting the web page on a display of the apparatus;

monitoring user input to determine a mode of interaction used by a user when interacting, via the apparatus, with at least one input field of a web page;

storing information indicating the determined mode of interaction used by the user when interacting with the at least one input field;

evaluating, by the computer processor, a user modal preference based, at least in part, on the stored information indicating the determined mode of interaction used by the user when interacting with the at least one input field; and

sending an indication of the user modal preference from the apparatus to the web server via at least one communications interface.

14. The computer-readable medium of claim 13, wherein the method further comprises presenting the multimodal application on a mobile device.

15. The computer-readable medium of claim 13, wherein evaluating the user modal preference further comprises determining that the user modal preference is the user modal preference indicated in the stored information.

16. The computer-readable medium of claim 13, wherein the method further comprises:

receiving, from the web server, multimodal content for the multimodal application configured based, at least in part, on the user modal preference; and

presenting the multimodal content via the apparatus.

17. The computer-readable medium of claim 13, wherein the method further comprises:

receiving a configuration value indicating the user modal preference; and

storing the configuration value;

wherein evaluating the user modal preference comprises retrieving the stored configuration value.

18. The computer-readable medium of claim 13, wherein storing information indicating the determined mode of interaction used by the user when interacting with the at least one input field comprises sending the information indicating the determined mode of interaction to a storage device connected to the apparatus via at least one network.