Enabling dynamic voiceXML in an X+V page of a multimodal application

• US 7,945,851 B2
• Filed: 03/14/2007
• Issued: 05/17/2011
• Est. Priority Date: 03/14/2007
• Status: Expired due to Fees

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Pike, Hilary A., White, Marc T., Seacat, Lisa A., Cross, Charles W. Jr.
• Primary Examiner(s): Basehoar; Adam L
• Assistant Examiner(s): SCOTT, MICHAEL E
• Application Number:
  US11/685,878
• Publication Number:
  US 20080228495A1
• Time in Patent Office: 1,525 Days
• Field of Search: 704/260, 704/270, 715/223, 715/221, 715/222, 715/234
• US Class Current: 715/223
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: H04M 2203/251 where a voice mode or a visual mode can be used interchangeably
• CPC Class Codes: H04M 2207/40 terminals with audio html browser
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

Enabling dynamic VoiceXML in an X+V page of a multimodal application implemented with the multimodal application operating in a multimodal browser on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, the multimodal application operatively coupled to a VoiceXML interpreter, including representing by the multimodal browser an XML element of a VoiceXML dialog of the X+V page as an ECMAScript object, the XML element comprising XML content; storing by the multimodal browser the XML content of the XML element in an attribute of the ECMAScript object; and accessing the XML content of the XML element in the attribute of the ECMAScript object from an ECMAScript script in the X+V page.

Claims

1. A method of enabling dynamic VoiceXML in an X+V page of a multimodal application, wherein the multimodal application operates in a multimodal browser on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, the multi modal application operatively coupled to a VoiceXML interpreter, the method comprising:

representing, by the multimodal browser, an XML element of a VoiceXML dialog of the X+V page as an ECMAScript object, the XML element comprising XML content, the XML content comprising text defining the XML element in the X+V page;

storing, by the multimodal browser, the XML content of the XML element in an attribute of the ECMAScript object, wherein the storing comprises storing in the attribute of the ECMAScript object at least a portion of the text defining the XML element in the X+V page; and

accessing the XML content of the XML element in the attribute of the ECMAScript object from an ECMAScript script in the X+V page, wherein the XML element comprises a VoiceXML <

field>

element of the VoiceXML dialog, and wherein the accessing comprises accessing the VoiceXML <

field>

element in the attribute of the ECMAScript object.

2. The method of claim 1 further comprising making XML content of XML elements stored in an attribute of an ECMAScript object accessible to ECMAScript scripts in X+V pages, including providing multimodal browser support for:

at least one native accessor function in ECMAScript objects for an attribute of an ECMAScript object that stores the XML content of an XML element of an X+V page; and

an ECMAScript infix assignment operator that is operable on the attribute of an ECMAScript object that stores the XML content of an XML element of an X+V page.

3. The method of claim 1, wherein accessing the XML content of the XML element in the attribute of the ECMAScript object further comprises updating the XML content of the XML element of the VoiceXML dialog with new XML content, the updating including:

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the XML element by replacing with the new XML content the contents of the attribute that stores the XML content of the XML element; and

if such interpretation is in process, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing the XML content of the XML element by replacing with the new XML content the contents of the attribute that stores the XML content of the XML element.

4. The method of claim 1, wherein:

representing an XML element of a VoiceXML dialog as an ECMAScript object further comprises representing the VoiceXML dialog as an ECMAScript object;

storing the XML content of the XML element in an attribute of the ECMAScript object further comprises storing the XML content of the VoiceXML dialog in an attribute of the ECMAScript object; and

accessing the XML content of the XML element in the attribute further comprises updating the XML content of the VoiceXML dialog with new XML content, the updating including;

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the VoiceXML dialog by replacing with the new XML content the contents of the attribute that stores the XML content of the VoiceXML dialog; and

if such interpretation is in process, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing with the new XML content the contents of the attribute that stores the XML content of the VoiceXML dialog.

5. The method of claim 1, wherein:

representing an XML element of a VoiceXML dialog as an ECMAScript object further comprises representing a VoiceXML <

field>

element of the VoiceXML dialog as an ECMAScript object;

storing the XML content of the XML element in an attribute of the ECMAScript object further comprises storing the XML content of the VoiceXML <

field>

element in an attribute of the ECMAScript object; and

accessing the XML content of the XML element in the attribute further comprises updating the XML content of the VoiceXML <

field>

element with new XML content, the updating including;

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the VoiceXML <

field>

element by replacing with the new XML content the contents of the attribute that stores the XML content of the VoiceXML <

field>

element;

if such interpretation is in process, determining whether the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element;

if the VoiceXML interpreter is not listening for speech input to the VoiceXML <

field>

element, replacing the XML content of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the VoiceXML <

field>

element with the new XML content; and

if the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing the XML content of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the VoiceXML <

field>

element with the new XML content.

6. The method of claim 1, wherein:

representing an XML element of a VoiceXML dialog as an ECMAScript object further comprises representing a component of a VoiceXML <

field>

element of the VoiceXML dialog as an ECMAScript object;

storing the XML content of the XML element in an attribute of the ECMAScript object further comprises storing the XML content of the component of the VoiceXML <

field>

element in an attribute of the ECMAScript object; and

accessing the XML content of the XML element in the attribute further comprises updating the XML content of the component of the VoiceXML <

field>

element with new XML content, the updating including;

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the component of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the component of the VoiceXML <

field>

element with the new XML content;

if such interpretation is in process, determining whether the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element;

if the VoiceXML interpreter is not listening for speech input to the VoiceXML <

field>

element, replacing the XML content of the component of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the component of the VoiceXML <

field>

element with the new XML content; and

if the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing the XML content of the component of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the component of the VoiceXML <

field>

element with the new XML content.

7. The method of claim 1, wherein the accessing comprises altering the at least a portion of the text defining the XML element in the X+V page, as stored in the attribute of the ECMAScript object, while the multimodal application is executing.

8. Apparatus for enabling dynamic VoiceXML in an X+V page of a multimodal application, wherein the multimodal application operates in a multimodal browser on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, the multimodal application operatively coupled to a VoiceXML interpreter, the apparatus comprising a computer processor and a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions capable of:

representing, by the multimodal browser, an XML element of a VoiceXML dialog of the X+V page as an ECMAScript object, the XML element comprising XML content, the XML content comprising text defining the XML element in the X+V page;

storing, by the multimodal browser, the XML content of the XML element in an attribute of the ECMAScript object, wherein the storing comprises storing in the attribute of the ECMAScript object at least a portion of the text defining the XML element in the X+V page; and

accessing the XML content of the XML element in the attribute of the ECMAScript object from an ECMAScript script in the X+V page, wherein the XML element comprises a VoiceXML <

field>

element of the VoiceXML dialog, and wherein the accessing comprises accessing the VoiceXML <

field>

element in the attribute of the ECMAScript object.

9. The apparatus of claim 8 further comprising computer program instructions capable of making XML content of XML elements stored in an attribute of an ECMAScript object accessible to ECMAScript scripts in X+V pages, including providing multimodal browser support for:

at least one native accessor function in ECMAScript objects for an attribute of an ECMAScript object that stores the XML content of an XML element of an X+V page; and

an ECMAScript infix assignment operator that is operable on the attribute of an ECMAScript object that stores the XML content of an XML element of an X+V page.

10. The apparatus of claim 8, wherein accessing the XML content of the XML element in the attribute of the ECMAScript object further comprises updating the XML content of the XML element of the VoiceXML dialog with new XML content, the updating including:

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the XML element by replacing with the new XML content the contents of the attribute that stores the XML content of the XML element; and

if such interpretation is in process, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing the XML content of the XML element by replacing with the new XML content the contents of the attribute that stores the XML content of the XML element.

11. The apparatus of claim 8, wherein:

representing an XML element of a VoiceXML dialog as an ECMAScript object further comprises representing the VoiceXML dialog as an ECMAScript object;

storing the XML content of the XML element in an attribute of the ECMAScript object further comprises storing the XML content of the VoiceXML dialog in an attribute of the ECMAScript object; and

accessing the XML content of the XML element in the attribute further comprises updating the XML content of the VoiceXML dialog with new XML content, the updating including;

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the VoiceXML dialog by replacing with the new XML content the contents of the attribute that stores the XML content of the VoiceXML dialog; and

if such interpretation is in process, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing with the new XML content the contents of the attribute that stores the XML content of the VoiceXML dialog.

12. The apparatus of claim 8, wherein:

representing an XML element of a VoiceXML dialog as an ECMAScript object further comprises representing a VoiceXML <

field>

element of the VoiceXML dialog as an ECMAScript object;

storing the XML content of the XML element in an attribute of the ECMAScript object further comprises storing the XML content of the VoiceXML <

field>

element in an attribute of the ECMAScript object; and

accessing the XML content of the XML element in the attribute further comprises updating the XML content of the VoiceXML <

field>

element with new XML content, the updating including;

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the VoiceXML <

field>

element by replacing with the new XML content the contents of the attribute that stores the XML content of the VoiceXML <

field>

element;

if such interpretation is in process, determining whether the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element;

if the VoiceXML interpreter is not listening for speech input to the VoiceXML <

field>

element, replacing the XML content of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the VoiceXML <

field>

element with the new XML content; and

if the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing the XML content of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the VoiceXML <

field>

element with the new XML content.

13. The apparatus of claim 8, wherein:

representing an XML element of a VoiceXML dialog as an ECMAScript object further comprises representing a component of a VoiceXML <

field>

element of the VoiceXML dialog as an ECMAScript object;

storing the XML content of the XML element in an attribute of the ECMAScript object further comprises storing the XML content of the component of the VoiceXML <

field>

element in an attribute of the ECMAScript object; and

accessing the XML content of the XML element in the attribute further comprises updating the XML content of the component of the VoiceXML <

field>

element with new XML content, the updating including;

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the component of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the component of the VoiceXML <

field>

element with the new XML content;

if such interpretation is in process, determining whether the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element;

if the VoiceXML interpreter is not listening for speech input to the VoiceXML <

field>

element, replacing the XML content of the component of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the component of the VoiceXML <

field>

element with the new XML content; and

if the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing the XML content of the component of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the component of the VoiceXML <

field>

element with the new XML content.

14. The apparatus of claim 8, wherein the accessing comprises altering the at least a portion of the text defining the XML element in the X+V page, as stored in the attribute of the ECMAScript object, while the multimodal application is executing.

15. A computer program product comprising at least one recordable computer-readable medium encoded with computer program instructions for enabling dynamic VoiceXML in an X+V page of a multimodal application capable of operating in a multimodal browser on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, the multimodal application capable of operatively coupling to a VoiceXML interpreter, the computer program instructions being capable of:

representing, by the multimodal browser, an XML element of a VoiceXML dialog of the X+V page as an ECMAScript object, the XML element comprising XML content, the XML content comprising text defining the XML element in the X+V page;

storing, by the multimodal browser, the XML content of the XML element in an attribute of the ECMAScript object, wherein the storing comprises storing in the attribute of the ECMAScript object at least a portion of the text defining the XML element in the X+V page; and

accessing the XML content of the XML element in the attribute of the ECMAScript object from an ECMAScript script in the X+V page, wherein the XML element comprises a VoiceXML <

field>

element of the VoiceXML dialog, and wherein the accessing comprises accessing the VoiceXML <

field>

element in the attribute of the ECMAScript object.

16. The computer program product of claim 15, wherein the computer program instructions are further capable of making XML content of XML elements stored in an attribute of an ECMAScript object accessible to ECMAScript scripts in X+V pages, including providing multimodal browser support for:

at least one native accessor function in ECMAScript objects for an attribute of an ECMAScript object that stores the XML content of an XML element of an X+V page; and

an ECMAScript infix assignment operator that is operable on the attribute of an ECMAScript object that stores the XML content of an XML element of an X+V page.

17. The computer program product of claim 15, wherein accessing the XML content of the XML element in the attribute of the ECMAScript object further comprises updating the XML content of the XML element of the VoiceXML dialog with new XML content, the updating including:

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the XML element by replacing with the new XML content the contents of the attribute that stores the XML content of the XML element; and

if such interpretation is in process, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing the XML content of the XML element by replacing with the new XML content the contents of the attribute that stores the XML content of the XML element.

18. The computer program product of claim 15, wherein:

representing an XML element of a VoiceXML dialog as an ECMAScript object further comprises representing the VoiceXML dialog as an ECMAScript object;

storing the XML content of the XML element in an attribute of the ECMAScript object further comprises storing the XML content of the VoiceXML dialog in an attribute of the ECMAScript object; and

accessing the XML content of the XML element in the attribute further comprises updating the XML content of the VoiceXML dialog with new XML content, the updating including;

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the VoiceXML dialog by replacing with the new XML content the contents of the attribute that stores the XML content of the VoiceXML dialog; and

if such interpretation is in process, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing with the new XML content the contents of the attribute that stores the XML content of the VoiceXML dialog.

19. The computer program product of claim 15, wherein:

representing an XML element of a VoiceXML dialog as an ECMAScript object further comprises representing a VoiceXML <

field>

element of the VoiceXML dialog as an ECMAScript object;

storing the XML content of the XML element in an attribute of the ECMAScript object further comprises storing the XML content of the VoiceXML <

field>

element in an attribute of the ECMAScript object; and

accessing the XML content of the XML element in the attribute further comprises updating the XML content of the VoiceXML <

field>

element with new XML content, the updating including;

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the VoiceXML <

field>

element by replacing with the new XML content the contents of the attribute that stores the XML content of the VoiceXML <

field>

element;

if such interpretation is in process, determining whether the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element;

if the VoiceXML interpreter is not listening for speech input to the VoiceXML <

field>

element, replacing the XML content of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the VoiceXML <

field>

element with the new XML content; and

if the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing the XML content of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the VoiceXML <

field>

element with the new XML content.

20. The computer program product of claim 15, wherein:

representing an XML element of a VoiceXML dialog as an ECMAScript object further comprises representing a component of a VoiceXML <

field>

element of the VoiceXML dialog as an ECMAScript object;

storing the XML content of the XML element in an attribute of the ECMAScript object further comprises storing the XML content of the component of the VoiceXML <

field>

element in an attribute of the ECMAScript object; and

accessing the XML content of the XML element in the attribute further comprises updating the XML content of the component of the VoiceXML <

field>

element with new XML content, the updating including;

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the component of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the component of the VoiceXML <

field>

element with the new XML content;

if such interpretation is in process, determining whether the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element;

if the VoiceXML interpreter is not listening for speech input to the VoiceXML <

field>

element, replacing the XML content of the component of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the component of the VoiceXML <

field>

element with the new XML content; and

if the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing the XML content of the component of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the component of the VoiceXML <

field>

element with the new XML content.

21. The computer program product of claim 15, wherein the accessing comprises altering the at least a portion of the text defining the XML element in the X+V page, as stored in the attribute of the ECMAScript object, while the multimodal application is executing.