Methods and Systems for Authoring of Mixed-Initiative Multi-Modal Interactions and Related Browsing Mechanisms

• US 20080034032A1
• Filed: 10/12/2007
• Published: 02/07/2008
• Est. Priority Date: 05/28/2002
• Status: Active Grant

• Current Assignee: Alpana Tiwari, Jennifer Healey, Rafah Hosn, Stephane Maes, Thiruvilvama Lai Raman
• Original Assignee: Alpana Tiwari, Jennifer Healey, Rafah Hosn, Stephane Maes, Thiruvilvama Lai Raman
• Inventors: Healey, Jennifer, Hosn, Rafah, Raman, Thiruvilvama Lai, Maes, Stephane, Tiwari, Alpana
• Granted Patent: US 8,572,209 B2
• Time in Patent Office: Days
• US Class Current: 709/203
• CPC Class Codes: G06F 8/00 Arrangements for software engineering testing or debugging G06F11/36; administrative, planning or organisation aspects of software project management G06Q10/06

Abstract

Application authoring techniques, and information browsing mechanisms associated therewith, which employ programming in association with mixed-initiative multi-modal interactions and natural language understanding for use in dialog systems. Also, a conversational browsing architecture is provided for use with these and other authoring techniques.

Claims

1. A conversational browser apparatus for providing access to an application by a user, the apparatus comprising:

an interaction manager;

at least one modality based browser operatively coupled to the interaction manager, the modality based browser being a voice-based browser; and

a dialog manager operatively coupled to the interaction manager;

wherein interaction between the user, the interaction manager, the voice-based browser and the the dialog manager comprises;

(i) the interaction manager obtains the application from a server in response to a user request, and passes a representation of the application to the dialog manger;

(ii) the dialog manager determines at least one of a focus of the application and an intent of the user in accordance with the representation of the application, and informs the interaction manager of at least one of the focus of the application and the intent of the user; and

(iii) the interaction manager then designates a particular portion of the application for presentation to the user in accordance with the voice-based browser based on the determination of at least one of the focus of the application and the intent of the user, the portion comprising at least a prompt and an input field, wherein the prompt is presented to the user and the input field specifies user input receivable in response to the prompt for use in redetermining at least one of the focus of the application and the intent of the user.

2. The apparatus of claim 1, wherein the interaction manager parses the application into a data model and one or more user interaction elements bound to the data model, generates a document object model (DOM) representation of at least the data model, and passes the DOM representation of the data model to the dialog manager.

3. The apparatus of claim 1, wherein the conversational browsing mechanism further comprises a natural language attribute value (AV) pair extractor operatively coupled between the voice-based browser and the interaction manager for extracting AV pairs associated with one or more user utterances for use by the dialog manager in determining at least one of the focus of the application and the intent of the user.

4. The apparatus of claim 1, wherein the designated portion further comprises a submit instruction to submit a result of the user input to the interaction manager.

5. The apparatus of claim 1, wherein the designated portion specifies a type of processing to be performed in accordance with at least one of the prompt and the user input.

6. The apparatus of claim 1, wherein the dialog manager operates independent of modalities.

7. The apparatus of claim 1, wherein the application server attempts to disambiguate an original user input provided thereto and, upon failing to do so, generates a sub-dialog to attempt to clarify the input such that the application server can reattempt to disambiguate the original user input based on additional input received from the user in response to the sub-dialog.

8. The apparatus of claim 1, wherein the designated portion is a Voice Extensible Markup Language (VoiceXML) snippet.

9. The apparatus of claim 1, wherein a menu is presented in an Extensible HyperText Markup Language (XHTML) frame or as part of a page of the application.

10. The apparatus of claim 1, wherein the voice-based browser supports Voice Extensible Markup Language (VoiceXML).

11. The apparatus of claim 1, wherein the conversational browsing mechanism further comprises a graphical user interface (GUI) based browser operatively coupled to the interaction manager for providing the user with the ability to interact with the application via at least a second modality.

12. The apparatus of claim 1, wherein communication between the interaction manager, the dialog manager and the application server is event-based.

13. The apparatus of claim 1, wherein a prompt is generated via a natural language generation (NLG) technique.

14. The apparatus of claim 1, wherein the application comprises forms and the dialog manager scores attribute value pairs generated from user input against each form and determines a top scoring form.

15. A conversational browser apparatus for providing access to an application by a user, the apparatus comprising:

an interaction manager function;

at least one modality based browser function operatively coupled to the interaction manager function; and

a dialog manager function operatively coupled to the interaction manager function;

wherein interaction between the user, the interaction manager, the browser and the the dialog manager functions comprises;

(i) the interaction manager obtains the application from a server in response to a user request, the application representing interactions that the user may have with the application as a data model and one or more user interaction elements that populate an application state of the data model and that are bound thereto;

(ii) the dialog manager receives attribute value pairs generated from user input and processes the attribute value pairs to populate the data model and determine a next interaction focus; and

(iii) upon determination of the next interaction focus by the dialog manager, the application state and context are updated and an output is generated by the interaction manager and provided to the user for interaction via the browser.

16. A method of authoring an application accessible by a user in accordance with a conversational browser, the method comprising the steps of:

an interaction manager of the conversational browser obtaining the application from a server in response to a user request, the application representing interactions that the user may have with the application as a data model and one or more user interaction elements that populate an application state of the data model and that are bound thereto;

a dialog manager of the conversational browser receiving attribute value pairs generated from user input and processing the attribute value pairs to populate the data model and determine a next interaction focus; and

upon determination of the next interaction focus by the dialog manager, the application state and context being updated and an output being generated by the interaction manager and provided to the user for interaction via a browsing mechanism of the conversational browser.