Invoking tapered prompts in a multimodal application

• US 8,744,861 B2
• Filed: 03/01/2012
• Issued: 06/03/2014
• Est. Priority Date: 02/26/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Ativanichayaphong, Soonthorn, Cross, Charles W. Jr., McCobb, Gerald M.
• Primary Examiner(s): ROBERTS, SHAUN A
• Application Number:
  US13/410,103
• Publication Number:
  US 20120166201A1
• Time in Patent Office: 824 Days
• Field of Search: 704/270, 704/274, 704/275
• US Class Current: 704/275
• CPC Class Codes: H04M 1/72445 for supporting Internet browser applications
• CPC Class Codes: H04M 2203/251 where a voice mode or a visual mode can be used interchangeably
• CPC Class Codes: H04M 2207/40 terminals with audio html browser
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

Methods, apparatus, and computer program products are described for invoking tapered prompts in a multimodal application implemented with a multimodal browser and a multimodal application operating on a multimodal device supporting multiple modes of user interaction with the multimodal application, the modes of user interaction including a voice mode and one or more non-voice modes. Embodiments include identifying, by a multimodal browser, a prompt element in a multimodal application; identifying, by the multimodal browser, one or more attributes associated with the prompt element; and playing a speech prompt according to the one or more attributes associated with the prompt element.

Claims

1. A computer-implemented method for invoking a tapered prompt comprising a plurality of prompt elements, each prompt element comprising a voice component and a non-voice component, the method comprising acts of:

selecting a first voice style for the voice component of a first prompt element in the plurality of prompt elements of the tapered prompt, wherein the voice component of the first prompt element solicits requested information from a user;

selecting, in conjunction with selecting the first voice style, a first non-voice style for the non-voice component of the first prompt element, wherein the non-voice component of the first prompt element solicits the same requested information as the voice component of the first prompt element;

receiving voice input provided by the user in response to the first prompt element;

using at least one processor to process the voice input to determine whether the user provided the requested information; and

in response to determining that at least some of the requested information was not provided by the user, selecting a second voice style for the voice component of a second prompt element of the tapered prompt, and selecting, in conjunction with selecting the second voice style, a second non-voice style for the non-voice component of the second prompt element, wherein;

both the voice component and non-voice component of the second prompt element further solicit the at least some of the requested information from the user,

the second voice style is different from the first voice style, and

the second non-voice style is different from the first non-voice style.

2. The computer-implemented method of claim 1, wherein the act of using at least one processor to process the voice input comprises an act of:

performing speech recognition processing on the voice input to determine whether the voice input matches an expected response.

3. The computer-implemented method of claim 2, wherein the act of selecting the second voice style is further based on a number of times the user was previously prompted, but failed, to provide voice input matching an expected response.

4. The computer-implemented method of claim 2, further comprising an act of:

if it is determined that the voice input does not match an expected response, generating, as the second prompt element, a prompt that prompts the user to speak one of a plurality of expected responses and recites at least one of the plurality of expected responses, wherein the first prompt element comprises an open-ended request that does not recite any one of the plurality of expected responses.

5. The computer-implemented method of claim 1, wherein the non-voice component of the second prompt element is a textual component, and wherein the second non-voice style is a textual style having an intensity that matches an intensity of the second voice style.

6. The computer-implemented method of claim 1, further comprising an act of:

audibly rendering the voice component of the first prompt element in the first voice style.

7. At least one computer readable storage device having encoded thereon instructions that, when executed by at least one processor, perform a method for invoking a tapered prompt comprising a plurality of prompt elements, each prompt element comprising a voice component and a non-voice component, the method comprising acts of:

selecting a first voice style for the voice component of a first prompt element in the plurality of prompt elements of the tapered prompt, wherein the voice component of the first prompt element solicits requested information from a user;

selecting, in conjunction with selecting the first voice style, a first non-voice style for the non-voice component of the first prompt element, wherein the non-voice component of the first prompt element solicits the same requested information as the voice component of the first prompt element;

receiving voice input provided by the user in response to the first prompt element;

processing the voice input to determine whether the user provided the requested information; and

in response to determining that at least some of the requested information was not provided by the user, selecting a second voice style for the voice component of a second prompt element of the tapered prompt, and selecting, in conjunction with selecting the second voice style, a second non-voice style for the non-voice component of the second prompt element, wherein;

both the voice component and non-voice component of the second prompt element further solicit the at least some of the requested information from the user,

the second voice style is different from the first voice style, and

the second non-voice style is different from the first non-voice style.

8. The at least one computer readable storage device of claim 7, wherein the act of processing the voice input comprises an act of:

performing speech recognition processing on the voice input to determine whether the voice input matches an expected response.

9. The at least one computer readable storage device of claim 8, wherein the act of selecting the second voice style is further based on a number of times the user was previously prompted, but failed, to provide voice input matching an expected response.

10. The at least one computer readable storage device of claim 8, wherein the method further comprises an act of:

if it is determined that the voice input does not match an expected response, generating, as the second prompt element, a prompt that prompts the user to speak one of a plurality of expected responses and recites at least one of the plurality of expected responses, wherein the first prompt element comprises an open-ended request that does not recite any one of the plurality of expected responses.

11. The at least one computer readable storage device of claim 7, wherein the non-voice component of the second prompt element is a textual component, and wherein the second non-voice style is a textual style having an intensity that matches an intensity of the second voice style.

12. The at least one computer readable storage device of claim 7, wherein the method further comprises an act of:

audibly rendering the voice component of the first prompt element in the first voice style.

13. A system for invoking a tapered prompt comprising a plurality of prompt elements, each prompt element comprising a voice component and a non-voice component, the system comprising at least one processor programmed to:

select a first voice style for the voice component of a first prompt element in the plurality of prompt elements of the tapered prompt, wherein the voice component of the first prompt element solicits requested information from a user;

select, in conjunction with selecting the first voice style, a first non-voice style for the non-voice component of the first prompt element, wherein the non-voice component of the first prompt element solicits the same requested information as the voice component of the first prompt element;

receive voice input provided by the user in response to the first prompt element;

process the voice input to determine whether the user provided the requested information; and

in response to determining that at least some of the requested information was not provided by the user, select a second voice style for the voice component of a second prompt element of the tapered prompt, and select, in conjunction with selecting the second voice style, a second non-voice style for the non-voice component of the second prompt element, wherein;

both the voice component and non-voice component of the second prompt element further solicit the at least some of the requested information from the user,

the second voice style is different from the first voice style, and

the second non-voice style is different from the first non-voice style.

14. The system of claim 13, wherein the at least one processor is programmed to process the voice input at least in part by:

performing speech recognition processing on the voice input to determine whether the voice input matches an expected response.

15. The system of claim 14, wherein the at least one processor is programmed to select the second voice style further based on a number of times the user was previously prompted, but failed, to provide voice input matching an expected response.

16. The system of claim 14, wherein the at least one processor is further programmed to:

if it is determined that the voice input does not match an expected response, generate, as the second prompt element, a prompt that prompts the user to speak one of a plurality of expected responses and recites at least one of the plurality of expected responses, wherein the first prompt element comprises an open-ended request that does not recite any one of the plurality of expected responses.

17. The system of claim 13, wherein the non-voice component of the second prompt element is a textual component, and wherein the second non-voice style is a textual style having an intensity that matches an intensity of the second voice style.

18. The system of claim 13, wherein the at least one processor is further programmed to:

audibly render the voice component of the first prompt element in the first voice style.