Establishing a preferred mode of interaction between a user and a multimodal application

• US 8,145,493 B2
• Filed: 09/11/2006
• Issued: 03/27/2012
• Est. Priority Date: 09/11/2006
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Cross, Jr., Charles W., Pike, Hilary A.
• Primary Examiner(s): Vo, Huyen X.
• Application Number:
  US11/530,599
• Publication Number:
  US 20080065386A1
• Time in Patent Office: 2,024 Days
• Field of Search: 379/210.01, 379/88.04, 715/746, 715/205, 370/466, 704/270.1, 704/257, 704/270, 704 1- 10, 704251-255, 709/220
• US Class Current: 704/270
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: H04M 3/42153 by subscriber
• CPC Class Codes: H04M 3/4936 Speech interaction details speech recognition per se G10L15/00
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML
• CPC Class Codes: H04N 21/4126 The peripheral being portable, e.g. PDAs or mobile phones

Abstract

Establishing a preferred mode of interaction between a user and a multimodal application, including evaluating, by a multimodal application operating on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, user modal preference, and dynamically configuring multimodal content of the multimodal application in dependence upon the evaluation of user modal preference.

Claims

1. A method of establishing a preferred mode of interaction between a user and a multimodal application, the method comprising:

evaluating, with at least one processor of a multimodal device comprising a multimodal application supporting multiple modes of interaction including a voice mode and one or more non-voice modes, a user modal preference, wherein evaluating the user modal preference comprises determining a modal coefficient value based, at least in part, on one or more modes of interaction used by the user in at least one previous interaction with a plurality of voice-enabled and/or non-voice enabled data entry fields presented by the multimodal application and evaluating the user modal preference based, at least in part, on the modal coefficient value; and

dynamically configuring, with the at least one processor, multimodal content of the multimodal application in dependence on the user modal preference;

wherein the multimodal application comprises a plurality of static markup documents, wherein the plurality of static markup documents comprises documents having different proportions of support for voice mode and non-voice modes;

wherein dynamically configuring multimodal content of the multimodal application further comprises selecting from among the plurality of static markup documents, in dependence upon the value of the user modal preference, a markup document for presentation to a user as part of the multimodal application.

2. The method of claim 1, wherein evaluating the user modal preference further comprises:

selecting the user modal preference as one of the one or more modes of interaction used by the user in at least one previous interaction.

3. The method of claim 1 wherein evaluating user modal preference further comprises evaluating user modal preference in dependence upon a user-supplied configuration value of user modal preference.

4. The method of claim 1 wherein the multimodal application comprises a set of multimodal application markup documents organized in a sequence of presentation, and wherein dynamically configuring multimodal content of the multimodal application further comprises altering the sequence of presentation.

5. The method of claim 1, wherein the multimodal application specifies a dynamic server page capable of generating the static markup documents of the multimodal application that include varying proportions of support for voice mode and non-voice modes, and wherein dynamically configuring multimodal content of the multimodal application further comprises generating by the dynamic server page a static markup document for presentation to a user as part of the multimodal application, the static markup document characterized by proportions of support for voice and non-voice modes established by the dynamic server page in dependence upon the value of the user modal preference.

6. The method of claim 1, further comprising:

receiving a configuration value indicating the user modal preference; and

storing the configuration value;

wherein evaluating the user modal preference comprises retrieving the stored configuration value.

7. Apparatus for establishing a preferred mode of interaction between a user and a multimodal application, the apparatus comprising:

at least one computer processor; and

at least one computer memory operatively coupled to the at least one computer processor, the at least one computer memory storing computer program instructions, that when executed by the at least one computer processor perform a method comprising;

evaluating, by a multimodal application operating on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, a user modal preference, wherein evaluating the user modal preference comprises determining a modal coefficient value based, at least in part, on one or more modes of interaction used by the user in at least one previous interaction with a plurality of voice-enabled and/or non-voice enabled data entry fields presented by the multimodal application and evaluating the user modal preference based, at least in part, on the modal coefficient value; and

dynamically configuring with the at least one computer processor, multimodal content of the multimodal application in dependence on the user modal preference;

wherein the multimodal application comprises a plurality of static markup documents, wherein the plurality of static markup documents comprises documents having different proportions of support for voice mode and non-voice modes;

wherein dynamically configuring multimodal content of the multimodal application further comprises selecting among the plurality of static markup documents, in dependence upon the value of the user modal preference, a markup document for presentation to a user as part of the multimodal application.

8. The apparatus of claim 7 wherein evaluating the user modal preference method further comprises:

selecting the user modal preference as one of the one or more modes of interaction used by the user in at least one previous interaction.

9. The apparatus of claim 7 wherein evaluating the user modal preference further comprises evaluating the user modal preference in dependence upon a user-supplied configuration value of the user modal preference.

10. The apparatus of claim 7 wherein the multimodal application comprises a set of multimodal application markup documents organized in a sequence of presentation, and wherein dynamically configuring multimodal content of the multimodal application further comprises altering the sequence of presentation.

11. The apparatus of claim 7, wherein the multimodal application specifies a dynamic server page capable of generating the static markup documents of the multimodal application that include varying proportions of support for voice mode and non-voice modes, and wherein dynamically configuring multimodal content of the multimodal application further comprises generating by the dynamic server page a static markup document for presentation to a user as part of the multimodal application, the static markup document characterized by proportions of support for voice and non-voice modes established by the dynamic server page in dependence upon the value of the user modal preference.

12. An apparatus for establishing a preferred mode of interaction between a user and a multimodal application, the apparatus comprising:

at least one computer processor; and

at least one computer memory operatively coupled to the at least one computer processor, the at least one computer memory having disposed within it computer program instructions capable of;

evaluating, by a multimodal application operating on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, a user modal preference, wherein evaluating the user modal preference comprises determining a modal coefficient value based, at least in part, on one or more modes of interaction used by the user in at least one previous interaction with a plurality of voice-enabled and/or non-voice enabled data entry fields presented by the multimodal application and evaluating the user modal preference based, at least in part, on the modal coefficient value; and

dynamically configuring with the at least one computer processor, multimodal content of the multimodal application in dependence upon the evaluation of the user modal preference;

wherein the multimodal application specifies a dynamic server page capable of generating static markup documents of the multimodal application that include varying proportions of support for voice mode and non-voice modes;

wherein dynamically configuring multimodal content of the multimodal application further comprises generating by the dynamic server page a static markup document for presentation to a user as part of the multimodal application, the static markup document characterized by proportions of support for voice and non-voice modes established by the dynamic server page in dependence upon the value of the user modal preference.

13. The apparatus of claim 12, wherein the method further comprises:

receiving a configuration value indicating the user modal preference; and

storing the configuration value;

wherein evaluating the user modal preference comprises retrieving the stored configuration value.

14. A non-transitory computer-readable recordable medium encoded with computer program instructions that, when executed by a computer, perform a method of establishing a preferred mode of interaction between a user and a multimodal application, the method comprising:

evaluating, by the multimodal application operating on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, a user modal preference, wherein evaluating the user modal preference comprises determining a modal coefficient value based, at least in part, on one or more modes of interaction used by the user in at least one previous interaction with a plurality of voice-enabled and/or non-voice enabled data entry fields presented by the multimodal application and evaluating the user modal preference based, at least in part, on the modal coefficient value; and

dynamically configuring multimodal content of the multimodal application in dependence on the user modal preference;

wherein the multimodal application comprises a plurality of static markup documents, wherein the plurality of static markup documents comprises documents having different proportions of support for voice mode and non-voice modes;

wherein dynamically configuring multimodal content of the multimodal application further comprises selecting among the plurality of static markup documents, in dependence upon the value of the user modal preference, a markup document for presentation to a user as part of the multimodal application.

15. The computer-readable recordable medium of claim 14, wherein the method further comprises:

selecting the user modal preference as one of the one or more modes of interaction used by the user in at least one previous interaction.

16. The computer-readable recordable medium of claim 14 wherein evaluating the user modal preference further comprises evaluating the user modal preference in dependence upon a user-supplied configuration value of the user modal preference.

17. The computer-readable recordable medium of claim 14 wherein the multimodal application comprises a set of multimodal application markup documents organized in a sequence of presentation, and wherein dynamically configuring multimodal content of the multimodal application further comprises altering the sequence of presentation.

18. The computer-readable recordable medium of claim 14 wherein the multimodal application specifies a dynamic server page capable of generating the static markup documents of the multimodal application that include varying proportions of support for voice mode and non-voice modes, and wherein dynamically configuring multimodal content of the multimodal application further comprises generating by the dynamic server page a static markup document for presentation to a user as part of the multimodal application, the static markup document characterized by proportions of support for voice and non-voice modes established by the dynamic server page in dependence upon the value of the user modal preference.

19. The computer-recordable medium of claim 14, wherein the method further comprises:

receiving a configuration value indicating the user modal preference; and

storing the configuration value;

wherein evaluating the user modal preference comprises retrieving the stored configuration value.

20. A non-transitory computer-readable recordable medium encoded with computer program instructions that, when executed by a computer, perform a method of establishing a preferred mode of interaction between a user and a multimodal application, the method comprising:

evaluating, by the multimodal application operating on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, a user modal preference, wherein evaluating the user modal preference comprises determining a modal coefficient value based, at least in part, on one or more modes of interaction used by the user in at least one previous interaction with a plurality of voice-enabled and/or non-voice enabled data entry fields presented by the multimodal application and evaluating the user modal preference based, at least in part, on the modal coefficient value; and

dynamically configuring multimodal content of the multimodal application in dependence upon the evaluation of the user modal preference;

wherein the multimodal application specifies a dynamic server page capable of generating static markup documents of the multimodal application that include varying proportions of support for voice mode and non-voice modes;

wherein dynamically configuring multimodal content of the multimodal application further comprises generating by the dynamic server page a static markup document for presentation to a user as part of the multimodal application, the static markup document characterized by proportions of support for voice and non-voice modes established by the dynamic server page in dependence upon the value of the user modal preference.

21. A method of establishing a preferred mode of interaction between a user and a multimodal application, the method comprising:

evaluating, with at least one processor of a multimodal device comprising a multimodal application supporting multiple modes of interaction including a voice mode and one or more non-voice modes, a user modal preference, wherein evaluating the user modal preference comprises determining a modal coefficient value based, at least in part, on one or more modes of interaction used by the user in at least one previous interaction with a plurality of voice-enabled and/or non-voice enabled data entry fields presented by the multimodal application and evaluating the user modal preference based, at least in part, on the modal coefficient value; and

dynamically configuring, with the at least one processor, multimodal content of the multimodal application in dependence upon the evaluation of the user modal preference;

wherein the multimodal application specifies a dynamic server page capable of generating static markup documents of the multimodal application that include varying proportions of support for voice mode and non-voice modes;

wherein dynamically configuring multimodal content of the multimodal application further comprises generating by the dynamic server page a static markup document for presentation to a user as part of the multimodal application, the static markup document characterized by proportions of support for voice and non-voice modes established by the dynamic server page in dependence upon the value of the user modal preference.