Establishing a preferred mode of interaction between a user and a multimodal application

• US 8,494,858 B2
• Filed: 02/14/2012
• Issued: 07/23/2013
• Est. Priority Date: 09/11/2006
• Status: Expired due to Fees

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Cross, Charles W. Jr., Pike, Hilary A.
• Primary Examiner(s): Vo, Huyen X.
• Application Number:
  US13/372,581
• Publication Number:
  US 20120151326A1
• Time in Patent Office: 525 Days
• Field of Search: 704 1- 10, 704/251, 704/257, 704/270, 704/270.1, 704/275, 704/235, 704/255, 715/205, 715/746, 710/107
• US Class Current: 704/270
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: H04M 3/42153 by subscriber
• CPC Class Codes: H04M 3/4936 Speech interaction details speech recognition per se G10L15/00
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML
• CPC Class Codes: H04N 21/4126 The peripheral being portable, e.g. PDAs or mobile phones

Abstract

Establishing a preferred mode of interaction between a user and a multimodal application, including evaluating, by a multimodal application operating on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, user modal preference, and dynamically configuring multimodal content of the multimodal application in dependence upon the evaluation of user modal preference.

Claims

1. A method of establishing a preferred mode of interaction between a user and a multimodal application supporting multiple modes of interaction including a voice mode and one or more non-voice modes, the method comprising:

determining a mode of interaction used by the user in interacting with at least one input field of a web page presented by the multimodal application;

storing information indicating the determined mode of interaction used by the user in interacting with the at least one input field;

evaluating, with at least one processor, a user modal preference, wherein evaluating the user modal preference comprises stored information indicating the mode of interaction used by the user in interacting with the at least one input field; and

dynamically configuring, with the at least one processor, multimodal content of the multimodal application in dependence on the user modal preference.

2. The method of claim 1, wherein evaluating the user modal preference further comprises selecting the user modal preference as the mode of interaction indicated in the stored information.

3. The method of claim 1, wherein the multimodal application comprises a set of multimodal application markup documents organized in a sequence of presentation, and wherein dynamically configuring multimodal content of the multimodal application further comprises altering the sequence of presentation.

4. The method of claim 1, wherein the multimodal application comprises a plurality of static markup documents, wherein the plurality of static markup documents comprise documents having different proportions of support for voice mode and non-voice modes,

wherein dynamically configuring multimodal content of the multimodal application comprises selecting from among the plurality of static markup documents, in dependence upon the value of the user modal preference, a markup document for presentation to a user as part of the multimodal application.

5. The method of claim 1, wherein the multimodal application specifies a dynamic server page capable of generating the static markup documents of the multimodal application that include varying proportions of support for voice mode and non-voice modes,

wherein dynamically configuring multimodal content of the multimodal application further comprises generating by the dynamic server page a static markup document for presentation to a user as part of the multimodal application, the static markup document characterized by proportions of support for voice and non-voice modes established by the dynamic server page in dependence upon the value of the user modal preference.

6. The method of claim 1, further comprising:

receiving a configuration value indicating the user modal preference; and

storing the configuration value;

wherein evaluating the user modal preference comprises retrieving the stored configuration value.

7. An apparatus for establishing a preferred mode of interaction between a user and a multimodal application supporting multiple modes of interaction including a voice mode and one or more non-voice modes, the apparatus comprising:

at least one computer processor; and

at least one computer memory operatively coupled to the at least one computer processor, the at least one computer memory storing computer program instructions, that when executed by the at least one computer processor perform a method comprising;

determining a mode of interaction used by the user in interacting with at least one input field of a web page presented by the multimodal application;

storing information indicating the determined mode of interaction used by the user in interacting with the at least one input field;

evaluating a user modal preference, wherein evaluating the user modal preference comprises stored information indicating the mode of interaction used by the user in interacting with the at least one input field; and

dynamically configuring with the at least one computer processor, multimodal content of the multimodal application in dependence on the user modal preference.

8. The apparatus of claim 7, wherein evaluating the user modal preference further comprises selecting the user modal preference as the mode of interaction indicated in the stored information.

9. The apparatus of claim 7, wherein the multimodal application comprises a set of multimodal application markup documents organized in a sequence of presentation, and wherein dynamically configuring multimodal content of the multimodal application further comprises altering the sequence of presentation.

10. The apparatus of claim 7, wherein the multimodal application comprises a plurality of static markup documents, wherein the plurality of static markup documents comprise documents having different proportions of support for voice mode and non-voice modes,

wherein dynamically configuring multimodal content of the multimodal application comprises selecting from among the plurality of static markup documents, in dependence upon the value of the user modal preference, a markup document for presentation to a user as part of the multimodal application.

11. The apparatus of claim 7, wherein the multimodal application specifies a dynamic server page capable of generating static markup documents of the multimodal application that include varying proportions of support for voice mode and non-voice modes;

wherein dynamically configuring multimodal content of the multimodal application further comprises generating by the dynamic server page a static markup document for presentation to a user as part of the multimodal application, the static markup document characterized by proportions of support for voice and non-voice modes established by the dynamic server page in dependence upon the value of the user modal preference.

12. The apparatus of claim 7, wherein the method further comprises:

receiving a configuration value indicating the user modal preference; and

storing the configuration value;

wherein evaluating the user modal preference comprises retrieving the stored configuration value.

13. A non-transitory computer-readable medium encoded with computer program instructions that, when executed by a computer, perform a method of establishing a preferred mode of interaction between a user and a multimodal application supporting multiple modes of interaction including a voice mode and one or more non-voice modes, the method comprising:

determining a mode of interaction used by the user in interacting with at least one input field of a web page presented by the multimodal application;

storing information indicating the determined mode of interaction used by the user in interacting with the at least one input field;

evaluating a user modal preference, wherein evaluating the user modal preference comprises evaluating the stored information indicating the mode of interaction used by the user in interacting with the at least one input field; and

dynamically configuring multimodal content of the multimodal application in dependence on the user modal preference.

14. The computer-readable medium of claim 13, wherein the multimodal application comprises a plurality of static markup documents, wherein the plurality of static markup documents comprises documents having different proportions of support for voice mode and non-voice modes,

wherein dynamically configuring multimodal content of the multimodal application further comprises selecting among the plurality of static markup documents, in dependence upon the value of the user modal preference, a markup document for presentation to a user as part of the multimodal application.

15. The computer-readable medium of claim 13, wherein the multimodal application specifies a dynamic server page capable of generating static markup documents of the multimodal application that include varying proportions of support for voice mode and non-voice modes;

wherein dynamically configuring multimodal content of the multimodal application further comprises generating by the dynamic server page a static markup document for presentation to a user as part of the multimodal application, the static markup document characterized by proportions of support for voice and non-voice modes established by the dynamic server page in dependence upon the value of the user modal preference.

16. The computer-readable medium of claim 13, wherein the method further comprises selecting the user modal preference as the mode of interaction indicated in the stored information.

17. The computer-readable medium of claim 13 wherein the multimodal application comprises a set of multimodal application markup documents organized in a sequence of presentation, and wherein dynamically configuring multimodal content of the multimodal application further comprises altering the sequence of presentation.

18. The computer-readable medium of claim 13, wherein the method further comprises:

receiving a configuration value indicating the user modal preference; and

storing the configuration value;

wherein evaluating the user modal preference comprises retrieving the stored configuration value.