Invoking tapered prompts in a multimodal application

• US 8,150,698 B2
• Filed: 02/26/2007
• Issued: 04/03/2012
• Est. Priority Date: 02/26/2007
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Ativanichayaphong, Soonthorn, Cross, Charles W. Jr., McCobb, Gerald M.
• Primary Examiner(s): Smits, Talivaldis Ivars
• Assistant Examiner(s): Roberts, Shaun
• Application Number:
  US11/678,920
• Publication Number:
  US 20080208588A1
• Time in Patent Office: 1,863 Days
• Field of Search: 704/270, 704/275, 704/274
• US Class Current: 704/275
• CPC Class Codes: H04M 1/72445 for supporting Internet browser applications
• CPC Class Codes: H04M 2203/251 where a voice mode or a visual mode can be used interchangeably
• CPC Class Codes: H04M 2207/40 terminals with audio html browser
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

Methods, apparatus, and computer program products are described for invoking tapered prompts in a multimodal application implemented with a multimodal browser and a multimodal application operating on a multimodal device supporting multiple modes of user interaction with the multimodal application, the modes of user interaction including a voice mode and one or more non-voice modes. Embodiments include identifying, by a multimodal browser, a prompt element in a multimodal application; identifying, by the multimodal browser, one or more attributes associated with the prompt element; and playing a speech prompt according to the one or more attributes associated with the prompt element.

Claims

1. A method for invoking tapered prompts in a multimodal application, the method implemented with a multimodal browser and a multimodal application operating on a multimodal device supporting multiple modes of user interaction with the multimodal application, the modes of user interaction including a voice mode and one or more non-voice modes, the method comprising acts of:

identifying a first prompt element and a second prompt element of a tapered prompt associated with a multimodal application;

rendering, in a first style, a first multimodal prompt associated with the first prompt element, the first multimodal prompt comprising a first speech prompt and a first non-speech prompt, the first style comprising a first speech style for the first speech prompt and a first non-speech style for the first non-speech prompt; and

rendering, in a second style, a second multimodal prompt associated with the second prompt element, the second multimodal prompt comprising a second speech prompt and a second non-speech prompt, the second non-speech prompt having content that is different from content of the first non-speech prompt, the second style comprising a second speech style for the second speech prompt and a second non-speech style for the second non-speech prompt, the second speech style being different from the first speech style, the second non-speech style being different from the first non-speech style and being selected in conjunction with the second speech style.

2. The method of claim 1 wherein the first prompt element is associated with a class attribute;

and wherein the act of rendering the first multimodal prompt comprises;

identifying a cascading style sheet based at least in part on a value of the class attribute;

playing according to the cascading style sheet the first speech prompt; and

rendering according to the cascading style sheet the first non-speech prompt, the first non-speech prompt comprising one or more visual elements associated with the first speech prompt.

3. The method of claim 1 wherein the first prompt element is associated with a prompt counter shadow variable and a source expression attribute whose value depends upon a value of the prompt counter shadow variable;

and wherein the act of rendering the first multimodal prompt comprises identifying the first speech prompt based at least in part on the value of the source expression attribute.

4. The method of claim 3 wherein the value of the source expression attribute comprises a local URL determined based at least in part on the value of the prompt counter shadow variable, and wherein the first speech prompt is a predetermined speech prompt identified by the local URL.

5. The method of claim 3 wherein the value of the source expression attribute comprises a data encoded URL for invoking a remote prompt creation engine, and wherein the URL is data encoded with the value of the prompt counter shadow variable and the first speech prompt is dynamically created by the remote prompt creation engine in dependence upon the value of the prompt counter shadow variable.

6. The method of claim 3 wherein the value of the source expression attribute comprises a local URL for invoking a local function call and the first speech prompt is dynamically created by invoking the local function call parameterized with the value of the prompt counter shadow variable.

7. The method of claim 1, wherein rendering the second multimodal prompt is performed in response to receiving input from a user responding to the first multimodal prompt, the input from the user not matching a speech grammar of the multimodal application.

8. An apparatus for invoking tapered prompts in a multimodal application, the apparatus comprising a multimodal device supporting multiple modes of user interaction including a voice mode and one or more non-voice modes, the apparatus further comprising a computer processor and a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions that, when executed, perform a method comprising acts of:

identifying a first prompt element and a second prompt element of a tapered prompt associated with a multimodal application;

rendering, in a first style, a first multimodal prompt associated with the first prompt element, the first multimodal prompt comprising a first speech prompt and a first non-speech prompt, the first style comprising a first speech style for the first speech prompt and a first non-speech style for the first non-speech prompt; and

rendering, in a second style, a second multimodal prompt associated with the second prompt element, the second multimodal prompt comprising a second speech prompt and a second non-speech prompt, the second non-speech prompt having content that is different from content of the first non-speech prompt, the second style comprising a second speech style for the second speech prompt and a second non-speech style for the second non-speech prompt, the second speech style being different from the first speech style, the second non-speech style being different from the first non-speech style and being selected in conjunction with the second speech style.

9. The apparatus of claim 8 wherein the first prompt element is associated with a class attribute;

and wherein the act of rendering the first multimodal prompt further comprises;

identifying a cascading style sheet based at least in part on a value of the class attribute;

playing according to the cascading style sheet the first speech prompt; and

rendering according to the cascading style sheet the first non-speech prompt, the first non-speech prompt comprising one or more visual elements associated with the first speech prompt.

10. The apparatus of claim 8 wherein the first prompt element is associated with a prompt counter shadow variable and a source expression attribute whose value depends upon a value of the prompt counter shadow variable;

and wherein the act of rendering the first multimodal prompt comprises identifying the first speech prompt based at least in part on the value of the source expression attribute.

11. The apparatus of claim 10 wherein the value of the source expression attribute comprises a local URL determined based at least in part on the value of the prompt counter shadow variable, and wherein the speech prompt is a predetermined speech prompt identified by the local URL.

12. The apparatus of claim 10 wherein the value of the source expression attribute comprises a data encoded URL for invoking a remote prompt creation engine, and wherein the URL is data encoded with the value of the prompt counter shadow variable and the first speech prompt is dynamically created by the remote prompt creation engine in dependence upon the value of the prompt counter shadow variable.

13. The apparatus of claim 10 wherein the value of the source expression attribute comprises a local URL for invoking a local function call and the first speech prompt is dynamically created by invoking the local function call parameterized with the value of the prompt counter shadow variable.

14. The apparatus of claim 8, wherein rendering the second multimodal prompt is performed in response to receiving input from a user responding to the first multimodal prompt, the input from the user not matching a speech grammar of the multimodal application.

15. The apparatus of claim 8, wherein the multimodal device comprises the computer processor and the computer memory.

16. A computer program product for invoking tapered prompts in a multimodal application, the computer program product including a multimodal browser operating on a multimodal device supporting multiple modes of user interaction with the multimodal application, the modes of user interaction including a voice mode and one or more non-voice modes, the computer program product disposed upon a computer-readable recordable medium, the computer program product comprising computer program instructions that, when executed, perform a method comprising acts of:

identifying a first prompt element and a second prompt element of a tapered prompt associated with a multimodal application;

rendering, in a first style, a first multimodal prompt associated with the first prompt element, the first multimodal prompt comprising a first speech prompt and a first non-speech prompt, the first style comprising a first speech style for the first speech prompt and a first non-speech style for the first non-speech prompt; and

rendering, in a second style, a second multimodal prompt associated with the second prompt element, the second multimodal prompt comprising a second speech prompt and a second non-speech prompt, the second non-speech prompt having content that is different from content of the first non-speech prompt, the second style comprising a second speech style for the second speech prompt and a second non-speech style for the second non-speech prompt, the second speech style being different from the first speech style, the second non-speech style being different from the first non-speech style and being selected in conjunction with the second speech style.

17. The computer program product of claim 16 wherein the first prompt element is associate with a class attribute;

and wherein the act of rendering the first multimodal prompt further comprises;

identifying a cascading style sheet based at least in part on a value of the class attribute;

playing according to the cascading style sheet the first speech prompt; and

rendering according to the cascading style sheet the first non-speech prompt, the first non-speech prompt comprising one or more visual elements associated with the first speech prompt.

18. The computer program product of claim 16 wherein the first prompt element is associated with a prompt counter shadow variable and a source expression attribute whose value depends upon a value of the prompt counter shadow variable;

and wherein the act of rendering the first multimodal prompt comprises identifying the first speech prompt based at least in part on the value of the source expression attribute.

19. The computer program product of claim 18 wherein the value of the source expression attribute comprises a local URL determined based at least in part on the value of the prompt counter shadow variable, and wherein the speech prompt is a predetermined speech prompt identified by the local URL.

20. The computer program product of claim 18 wherein the value of the source expression attribute comprises a data encoded URL for invoking a remote prompt creation engine, and wherein the URL is data encoded with the value of the prompt counter shadow variable and the first speech prompt is dynamically created by the remote prompt creation engine in dependence upon the value of the prompt counter shadow variable.

21. The computer program product of claim 18 wherein the value of the source expression attribute comprises a local URL for invoking a local function call and the first speech prompt is dynamically created by invoking the local function call parameterized with the value of the prompt counter shadow variable.

22. The computer program product of claim 16, wherein rendering the second multimodal prompt is performed in response to receiving input from a user responding to the first multimodal prompt, the input from the user not matching a speech grammar of the multimodal application.

23. A method for invoking a tapered prompt associated with a multimodal application, the method being implemented on a multimodal device supporting multiple modes of user interaction with the multimodal application, the modes of user interaction including a voice mode and one or more non-voice modes, the method comprising acts of:

identifying a first prompt element and a second prompt element of the tapered prompt;

identifying a first value of at least one first attribute associated with the first prompt element;

rendering a first multimodal prompt associated with the first prompt element in a first style selected from a plurality of styles based at least in part on the first value of the at least one first attribute, the first multimodal prompt comprising a first speech prompt and a first textual prompt, the first style comprising a first speech style for the first speech prompt and a first textual style for the first textual prompt;

identifying a second value of at least one second attribute associated with the second prompt element; and

rendering a second multimodal prompt associated with the second prompt element in a second style selected from the plurality of styles based at least in part on the second value of the at least one second attribute, the second multimodal prompt comprising a second speech prompt and a second textual prompt, the second style comprising a second speech style for the second speech prompt and a second textual style for the second textual prompt, the second speech style being different from the first speech style, the second textual style being different from the first textual style and being selected to match the second speech style.

24. The method of claim 23, wherein the act of rendering the first multimodal prompt comprises:

identifying a cascading style sheet based at least in part on the first value of the at least one first attribute; and

rendering the first multimodal prompt according to the cascading style sheet.

25. The method of claim 23, wherein the at least one first attribute comprises a prompt counter shadow variable and a source expression attribute whose value depends upon a value of the prompt counter shadow variable, and wherein the method further comprises:

identifying the first speech prompt based at least in part on the value of the source expression attribute.

26. The method of claim 23, wherein the first textual style comprises a first font and the second textual style comprises a second font that is more prominent than the first font.