Enabling Dynamic VoiceXML In An X+ V Page Of A Multimodal Application

• US 20080228495A1
• Filed: 03/14/2007
• Published: 09/18/2008
• Est. Priority Date: 03/14/2007
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Pike, Hilary A., Seacat, Lisa A., Cross, Charles W. Jr., White, Marc T.
• Granted Patent: US 7,945,851 B2
• Time in Patent Office: Days
• US Class Current: 704/275
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: H04M 2203/251 where a voice mode or a visual mode can be used interchangeably
• CPC Class Codes: H04M 2207/40 terminals with audio html browser
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

Enabling dynamic VoiceXML in an X+V page of a multimodal application implemented with the multimodal application operating in a multimodal browser on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, the multimodal application operatively coupled to a VoiceXML interpreter, including representing by the multimodal browser an XML element of a VoiceXML dialog of the X+V page as an ECMAScript object, the XML element comprising XML content; storing by the multimodal browser the XML content of the XML element in an attribute of the ECMAScript object; and accessing the XML content of the XML element in the attribute of the ECMAScript object from an ECMAScript script in the X+V page.

Claims

1. A method of enabling dynamic VoiceXML in an X+V page of a multimodal application, the method implemented with the multimodal application operating in a multimodal browser on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, the multimodal application operatively coupled to a VoiceXML interpreter, the method comprising:

representing by the multimodal browser an XML element of a VoiceXML dialog of the X+V page as an ECMAScript object, the XML element comprising XML content;

storing by the multimodal browser the XML content of the XML element in an attribute of the ECMAScript object; and

accessing the XML content of the XML element in the attribute of the ECMAScript object from an ECMAScript script in the X+V page.

2. The method of claim 1 further comprising making XML content of XML elements stored in an attribute of an ECMAScript object accessible to ECMAScript scripts in X+V pages, including providing multimodal browser support for:

at least one native accessor function in ECMAScript objects for an attribute of an ECMAScript object that stores the XML content of an XML element of an X+V page; and

an ECMAScript infix assignment operator that is operable on the attribute of an ECMAScript object that stores the XML content of an XML element of an X+V page.

3. The method of claim 1 wherein accessing the XML content of the XML element in the attribute of the ECMAScript object further comprises updating the XML content of the XML element of the VoiceXML dialog with new XML content, including:

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the XML element by replacing with the new XML content the contents of the attribute that stores the XML content of the XML element; and

if such interpretation is in process, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing the XML content of the XML element by replacing with the new XML content the contents of the attribute that stores the XML content of XML element.

4. The method of claim 1 wherein:

representing an XML element of a VoiceXML dialog as an ECMAScript object further comprises representing the VoiceXML dialog as an ECMAScript object;

storing the XML content of the XML element in an attribute of the ECMAScript object further comprises storing the XML content of the VoiceXML dialog in an attribute of the ECMAScript object; and

accessing the XML content of the XML element in the attribute further comprises updating the XML content of the VoiceXML dialog with new XML content, including;

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the VoiceXML dialog by replacing with the new XML content the contents of the attribute that stores the XML content of the VoiceXML dialog; and

if such interpretation is in process, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing with the new XML content the contents of the attribute that stores the XML content of the VoiceXML dialog.

5. The method of claim 1 wherein:

representing an XML element of a VoiceXML dialog as an ECMAScript object further comprises representing a VoiceXML <

field>

element of the VoiceXML dialog as an ECMAScript object;

storing the XML content of the XML element in an attribute of the ECMAScript object further comprises storing the XML content of the VoiceXML <

field>

element in an attribute of the ECMAScript object; and

accessing the XML content of the attribute further comprises updating the XML content of the VoiceXML <

field>

element with new XML content, including;

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the VoiceXML <

field>

element by replacing with the new XML content the contents of the attribute that stores the XML content of the VoiceXML <

field>

element; and

if such interpretation is in process, determining whether the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element;

if the VoiceXML interpreter is not listening for speech input to the VoiceXML <

field>

element, replacing the XML content of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the VoiceXML <

field>

element with the new XML content; and

if the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing the XML content of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the VoiceXML <

field>

element with the new XML content.

6. The method of claim 1 wherein:

representing an XML element of a VoiceXML dialog as an ECMAScript object further comprises representing a component of a VoiceXML <

field>

element of the VoiceXML dialog as an ECMAScript object; and

storing the XML content of the XML element in an attribute of the ECMAScript object further comprises storing the XML content of the component of the VoiceXML <

field>

element in an attribute of the ECMAScript object; and

accessing the XML content of the attribute further comprises updating the XML content of the component of the VoiceXML <

field>

element with new XML content, including;

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the component of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the component of VoiceXML <

field>

element with the new XML content; and

if such interpretation is in process, determining whether the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element;

if the VoiceXML interpreter is not listening for speech input to the VoiceXML <

field>

element, replacing the XML content of the component of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the component of the VoiceXML <

field>

element with the new XML content; and

if the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing the XML content of the component of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the component of the VoiceXML <

field>

element with the new XML content.

7. Apparatus for enabling dynamic VoiceXML in an X+V page of a multimodal application, the apparatus implemented with the multimodal application operating in a multimodal browser on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, the multimodal application operatively coupled to a VoiceXML interpreter, the apparatus comprising a computer processor and a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions capable of:

representing by the multimodal browser an XML element of a VoiceXML dialog of the X+V page as an ECMAScript object, the XML element comprising XML content;

storing by the multimodal browser the XML content of the XML element in an attribute of the ECMAScript object; and

accessing the XML content of the XML element in the attribute of the ECMAScript object from an ECMAScript script in the X+V page.

8. The apparatus of claim 7 further comprising computer program instructions capable of making XML content of XML elements stored in an attribute of an ECMAScript object accessible to ECMAScript scripts in X+V pages, including providing multimodal browser support for:

at least one native accessor function in ECMAScript objects for an attribute of an ECMAScript object that stores the XML content of an XML element of an X+V page; and

an ECMAScript infix assignment operator that is operable on the attribute of an ECMAScript object that stores the XML content of an XML element of an X+V page.

9. The apparatus of claim 7 wherein accessing the XML content of the XML element in the attribute of the ECMAScript object further comprises updating the XML content of the XML element of the VoiceXML dialog with new XML content, including:

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the XML element by replacing with the new XML content the contents of the attribute that stores the XML content of the XML element; and

if such interpretation is in process, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing the XML content of the XML element by replacing with the new XML content the contents of the attribute that stores the XML content of XML element.

10. The apparatus of claim 7 wherein:

representing an XML element of a VoiceXML dialog as an ECMAScript object further comprises representing the VoiceXML dialog as an ECMAScript object;

storing the XML content of the XML element in an attribute of the ECMAScript object further comprises storing the XML content of the VoiceXML dialog in an attribute of the ECMAScript object; and

accessing the XML content of the XML element in the attribute further comprises updating the XML content of the VoiceXML dialog with new XML content, including;

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the VoiceXML dialog by replacing with the new XML content the contents of the attribute that stores the XML content of the VoiceXML dialog; and

if such interpretation is in process, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing with the new XML content the contents of the attribute that stores the XML content of the VoiceXML dialog.

11. The apparatus of claim 7 wherein:

representing an XML element of a VoiceXML dialog as an ECMAScript object further comprises representing a VoiceXML <

field>

element of the VoiceXML dialog as an ECMAScript object;

storing the XML content of the XML element in an attribute of the ECMAScript object further comprises storing the XML content of the VoiceXML <

field>

element in an attribute of the ECMAScript object; and

accessing the XML content of the attribute further comprises updating the XML content of the VoiceXML <

field>

element with new XML content, including;

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the VoiceXML <

field>

element by replacing with the new XML content the contents of the attribute that stores the XML content of the VoiceXML <

field>

element; and

if such interpretation is in process, determining whether the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element;

if the VoiceXML interpreter is not listening for speech input to the VoiceXML <

field>

element, replacing the XML content of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the VoiceXML <

field>

element with the new XML content; and

if the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing the XML content of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the VoiceXML <

field>

element with the new XML content.

12. The apparatus of claim 7 wherein:

representing an XML element of a VoiceXML dialog as an ECMAScript object further comprises representing a component of a VoiceXML <

field>

element of the VoiceXML dialog as an ECMAScript object; and

storing the XML content of the XML element in an attribute of the ECMAScript object further comprises storing the XML content of the component of the VoiceXML <

field>

element in an attribute of the ECMAScript object; and

accessing the XML content of the attribute further comprises updating the XML content of the component of the VoiceXML <

field>

element with new XML content, including;

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the component of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the component of VoiceXML <

field>

element with the new XML content; and

if such interpretation is in process, determining whether the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element;

if the VoiceXML interpreter is not listening for speech input to the VoiceXML <

field>

element, replacing the XML content of the component of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the component of the VoiceXML <

field>

element with the new XML content; and

if the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing the XML content of the component of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the component of the VoiceXML <

field>

element with the new XML content.

13. A computer program product for enabling dynamic VoiceXML in an X+V page of a multimodal application, the computer program product comprising a multimodal application capable of operating in a multimodal browser on a multimodal device supporting multiple modes of interaction including a voice mode and one or more non-voice modes, the multimodal application capable of operatively coupling to a VoiceXML interpreter, the computer program product disposed upon a computer-readable, signal-bearing medium, the computer program product comprising computer program instructions capable of:

representing by the multimodal browser an XML element of a VoiceXML dialog of the X+V page as an ECMAScript object, the XML element comprising XML content;

storing by the multimodal browser the XML content of the XML element in an attribute of the ECMAScript object; and

accessing the XML content of the XML element in the attribute of the ECMAScript object from an ECMAScript script in the X+V page.

14. The computer program product of claim 13 wherein the computer-readable, signal-bearing medium comprises a recordable medium.

15. The computer program product of claim 13 wherein the computer-readable, signal-bearing medium comprises a transmission medium.

16. The computer program product of claim 13 further comprising computer program instructions capable of making XML content of XML elements stored in an attribute of an ECMAScript object accessible to ECMAScript scripts in X+V pages, including providing multimodal browser support for:

at least one native accessor function in ECMAScript objects for an attribute of an ECMAScript object that stores the XML content of an XML element of an X+V page; and

an ECMAScript infix assignment operator that is operable on the attribute of an ECMAScript object that stores the XML content of an XML element of an X+V page.

17. The computer program product of claim 13 wherein accessing the XML content of the XML element in the attribute of the ECMAScript object further comprises updating the XML content of the XML element of the VoiceXML dialog with new XML content, including:

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the XML element by replacing with the new XML content the contents of the attribute that stores the XML content of the XML element; and

if such interpretation is in process, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing the XML content of the XML element by replacing with the new XML content the contents of the attribute that stores the XML content of XML element.

18. The computer program product of claim 13 wherein:

representing an XML element of a VoiceXML dialog as an ECMAScript object further comprises representing the VoiceXML dialog as an ECMAScript object;

storing the XML content of the XML element in an attribute of the ECMAScript object further comprises storing the XML content of the VoiceXML dialog in an attribute of the ECMAScript object; and

accessing the XML content of the XML element in the attribute further comprises updating the XML content of the VoiceXML dialog with new XML content, including;

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the VoiceXML dialog by replacing with the new XML content the contents of the attribute that stores the XML content of the VoiceXML dialog; and

if such interpretation is in process, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing with the new XML content the contents of the attribute that stores the XML content of the VoiceXML dialog.

19. The computer program product of claim 13 wherein:

representing an XML element of a VoiceXML dialog as an ECMAScript object further comprises representing a VoiceXML <

field>

element of the VoiceXML dialog as an ECMAScript object;

storing the XML content of the XML element in an attribute of the ECMAScript object further comprises storing the XML content of the VoiceXML <

field>

element in an attribute of the ECMAScript object; and

accessing the XML content of the attribute further comprises updating the XML content of the VoiceXML <

field>

element with new XML content, including;

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the VoiceXML <

field>

element by replacing with the new XML content the contents of the attribute that stores the XML content of the VoiceXML <

field>

element; and

if such interpretation is in process, determining whether the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element;

if the VoiceXML interpreter is not listening for speech input to the VoiceXML <

field>

element, replacing the XML content of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the VoiceXML <

field>

element with the new XML content; and

if the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing the XML content of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the VoiceXML <

field>

element with the new XML content.

20. The computer program product of claim 13 wherein:

representing an XML element of a VoiceXML dialog as an ECMAScript object further comprises representing a component of a VoiceXML <

field>

element of the VoiceXML dialog as an ECMAScript object; and

storing the XML content of the XML element in an attribute of the ECMAScript object further comprises storing the XML content of the component of the VoiceXML <

field>

element in an attribute of the ECMAScript object; and

accessing the XML content of the attribute further comprises updating the XML content of the component of the VoiceXML <

field>

element with new XML content, including;

determining whether VoiceXML interpretation of the VoiceXML dialog is in process;

if such interpretation is not in process, replacing the XML content of the component of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the component of VoiceXML <

field>

element with the new XML content; and

if such interpretation is in process, determining whether the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element;

if the VoiceXML interpreter is not listening for speech input to the VoiceXML <

field>

element, replacing the XML content of the component of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the component of the VoiceXML <

field>

element with the new XML content; and

if the VoiceXML interpreter is listening for speech input to the VoiceXML <

field>

element, canceling the VoiceXML interpretation of the VoiceXML dialog and replacing the XML content of the component of the VoiceXML <

field>

element by replacing the contents of the attribute that stores the XML content of the component of the VoiceXML <

field>

element with the new XML content.