Dynamically generating a vocal help prompt in a multimodal application

• US 8,086,463 B2
• Filed: 09/12/2006
• Issued: 12/27/2011
• Est. Priority Date: 09/12/2006
• Status: Expired due to Fees

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation, Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Ativanichayaphong, Soonthorn, Cross, Charles W. Jr., Jaramillo, David, Li, Yan
• Primary Examiner(s): YEN, ERIC L
• Application Number:
  US11/530,930
• Publication Number:
  US 20080065390A1
• Time in Patent Office: 1,932 Days
• Field of Search: 704270-275
• US Class Current: 704/275
• CPC Class Codes: G06F 9/453 Help systems
• CPC Class Codes: G09B 5/06 with both visual and audible presentation of the material to be studied
• CPC Class Codes: G10L 13/027 Concept to speech synthesisers; Generation of natural phrases from machine-based concepts generation of parameters for speech synthesis out of text G10L13/08
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

Dynamically generating a vocal help prompt in a multimodal application that include detecting a help-triggering event for an input element of a VoiceXML dialog, where the detecting is implemented with a multimodal application operating on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, the multimodal application is operatively coupled to a VoiceXML interpreter, and the multimodal application has no static help text. Dynamically generating a vocal help prompt in a multimodal application according to embodiments of the present invention typically also includes retrieving, by the VoiceXML interpreter from a source of help text, help text for an element of a speech recognition grammar, forming by the VoiceXML interpreter the help text into a vocal help prompt, and presenting by the multimodal application the vocal help prompt through a computer user interface to a user.

Claims

1. A method for dynamically generating a vocal help prompt in a multimodal application, the method comprising:

detecting a help-triggering event for an input element of a VoiceXML dialog, wherein the help-triggering event is selected from the group consisting of a request by a user for help, speech input that does not match any active grammar, and no speech input being received for a specified period of time, the detecting implemented with the multimodal application operating on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, the multimodal application operatively coupled to a VoiceXML interpreter;

retrieving, by the VoiceXML interpreter from a speech recognition grammar updated based on a changing information source, retrieved help text, wherein the retrieved help text includes first help text associated with at least one non-terminal element of the speech recognition grammar and second help text associated with at least one terminal element of the speech recognition grammar, wherein at least some of the retrieved help text is not hard-coded by a programmer of the multimodal application

generating, by the VoiceXML interpreter, a vocal help prompt based, at least in part, on the first help text and the second help text; and

presenting by the multimodal application the vocal help prompt through a computer user interface to a user.

2. The method of claim 1 wherein generating the vocal help prompt further comprises concatenating the first help text associated with the at least one non-terminal element of the grammar with the second help text associated with the at least one terminal element of the grammar.

3. The method of claim 1 wherein:

the grammar includes two or more alternative terminal elements; and

retrieving help text further comprises retrieving, for use as help text, text from a number of the alternative terminal elements, the number limited by a predefined maximum.

4. The method of claim 1 further comprising iteratively carrying out the steps of the method upon occurrence of additional help-triggering events, generating multiple levels of help.

5. The method of claim 1 wherein:

the grammar includes words that can be recognized by the VoiceXML interpreter.

6. The method of claim 1 wherein:

the speech recognition grammar was previously dynamically generated using text stored in a computer database.

7. Apparatus for dynamically generating a vocal help prompt in a multimodal application, the apparatus comprising:

a computer processor; and

a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions that, when executed by the computer processor, perform a method comprising;

detecting a help-triggering event for an input element of a VoiceXML dialog, wherein the help-triggering event is selected from the group consisting of a request by a user for help, speech input that does not match any active grammar, and no speech input being received for a specified period of time, the detecting implemented with the multimodal application operating on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, the multimodal application operatively coupled to a VoiceXML interpreter;

retrieving, by the VoiceXML interpreter from a speech recognition grammar updated based on a changing information source, retrieved help text, associated with at least one non-terminal element of the speech recognition grammar and at least one terminal element of the speech recognition grammar, wherein the at least one non-terminal element includes a reference to at least one other element of the speech recognition grammar, wherein at least some of the help text is not hard-coded by a programmer of the multimodal application.

forming, by the VoiceXML interpreter, the retrieved help text into a vocal help prompt; and

presenting by the multimodal application the vocal help prompt through a computer user interface to a user.

8. The apparatus of claim 7 wherein forming the help text into a vocal help prompt further comprises concatenating help text associated with the at least one non-terminal element of the grammar with help text associated with the at least one terminal element of the grammar.

9. The apparatus of claim 7 wherein:

the grammar includes two or more alternative terminal elements; and

retrieving help text further comprises retrieving, for use as help text, text from a number of the alternative terminal elements, the number limited by a predefined maximum.

10. The apparatus of claim 7 further comprising iteratively carrying out the steps of the method upon occurrence of additional help-triggering events, generating multiple levels of help.

11. The apparatus of claim 7 wherein:

the grammar includes words that can be recognized by the VoiceXML interpreter.

12. The apparatus of claim 7 wherein:

the speech recognition grammar was previously dynamically generated using text stored in a computer database.

13. A non-transitory computer-readable recordable medium encoded with a plurality of instructions that, when executed by a computer perform a method of dynamically generating a vocal help prompt in a multimodal application, the method comprising:

detecting a help-triggering event for an input element of a VoiceXML dialog, wherein the help-triggering event is selected from the group consisting of a request by a user for help, speech input that does not match any active grammar, and no speech input being received for a specified period of time, the detecting implemented with the multimodal application operating on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, the multimodal application operatively coupled to a VoiceXML interpreter;

retrieving, by the VoiceXML interpreter from a speech recognition grammar updated based on a changing information source, retrieved help text, wherein the retrieved help text includes first help text associated with at least one non-terminal element of the speech recognition grammar and second help text associated with at least one terminal element of the speech recognition grammar, wherein at least some of the retrieved help text is not hard-coded by a programmer of the multimodal application

generating, by the VoiceXML interpreter, a vocal help prompt based, at least in part, on the first help text and the second help text; and

presenting by the multimodal application the vocal help prompt through a computer user interface to a user.

14. The computer-readable recordable medium of claim 13 wherein generating the vocal help prompt further comprises concatenating the first help text associated with the at least one non-terminal element of the grammar with the second help text associated with the at least one terminal element of the grammar.

15. The computer-readable recordable medium of claim 13 further comprising iteratively carrying out the steps of the method upon occurrence of additional help-triggering events, generating multiple levels of help.