Context-sensitive interface widgets for multi-modal dialog systems

• US 20040122674A1
• Filed: 12/19/2002
• Published: 06/24/2004
• Est. Priority Date: 12/19/2002
• Status: Active Grant

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: AT&T Corporation (AT&T, Inc.)
• Inventors: Johnston, Michael, Bangalore, Srinivas
• Granted Patent: US 7,890,324 B2
• Time in Patent Office: Days
• US Class Current: 704/276
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A system and method of presenting widgets to a user during a multi-modal interactive dialog between a user and a computer is presented. The system controls the multi-modal dialog; and when user input would help to clarify or speed up the presentation of requested information, the system presents a temporary widget to the user to elicit the user input in this regard. The system presents the widget on a display screen at a position that will not interfere with the dialog. Various types of widgets are available, such as button widgets, sliders and confirmation widgets, depending on the type of information that the system requires.

Claims

1. In a multi-modal dialog system, a method of providing widgets to a user, comprising, after first user input and where further user input will clarify the first user input during a multi-modal dialog:

maintaining a current display screen context; and

presenting a widget on a display screen to elicit the further user input.

2. The method of claim 1, wherein the user may manipulate the widget using a stylus, mouse, or touch.

3. The method of claim 1, wherein the user may manipulate the widget via speech.

4. The method of claim 1, wherein the widget appears while the further user input is relevant to a current context of the multi-modal dialog.

5. The method of claim 1, wherein the widget appears at a location on the display screen that does not interfere with the multi-modal dialog.

6. The method of claim 5, wherein the multi-modal dialog system dynamically selects the location of the widget according to a multi-modal dialog context such that the widget does not cover relevant information on the display screen.

7. The method of claim 1, wherein the widget comprises a list of selectable buttons.

8. The method of claim 1, wherein the widget comprises a slider.

9. The method of claim 1, wherein the widget comprises arrow buttons.

10. A method of temporarily providing one of a plurality of widgets to a user in the course of a multi-modal dialog with a computer device, the method comprising:

after first user input, determining whether further user input would be advantageous before the computer device presents information to the user;

if further user input would be advantageous, selecting a widget from a plurality of widgets, the selected widget being associated with the further user input; and

presenting the selected widget on a display for receiving the further user input.

11. The method of claim 10, wherein the plurality of widgets comprises at least button widgets and slider widgets.

12. The method of claim 10, wherein the computer device dynamically generates the content of the selected widget according to the required further user input.

13. The method of claim 10, further comprising, when the user provides the further input, removing the widget from the display.

14. A multi-modal dialog system comprising:

a multi-modal interface module that receives multi-modal input from a user and provides multi-modal information to the user; and

a widget control module that dynamically presents widgets on a display screen to facilitate a multi-modal dialog between the user and the multi-modal dialog system.

15. The multi-modal dialog system of claim 14, wherein the widget control module selects a widget from a plurality of widgets.

16. The multi-modal dialog system of claim 15, wherein the plurality of widgets comprises at least a button widget and a slider widget.

17. The multi-modal dialog system of claim 14, wherein the widget control module only presents widgets in the course of the multi-modal dialog when the multi-modal dialog system receives first user input and further user input could refine the first user input.

18. The multi-modal dialog system of claim 14, wherein the user can provide user input via the widget by stylus, mouse control, or touch.

19. The multi-modal dialog system of claim 14, wherein the user can provide user input in response to the widget via speech.

20. The multi-modal dialog system of claim 14, wherein the widget module further presents widgets on the display screen at a location that does not interfere with the presentation of multi-modal information.

21. The multi-modal dialog system of claim 14, wherein the widgets are transparent.

22. The multi-modal dialog system of claim 14, wherein the widget module dynamically selects a location of the widget according to content on the display screen.

23. The multi-modal dialog system of claim 14, wherein the user can control the appearance of the widgets during the multi-modal dialog.