Context-sensitive interface widgets for multi-modal dialog systems

• US 7,890,324 B2
• Filed: 12/19/2002
• Issued: 02/15/2011
• Est. Priority Date: 12/19/2002
• Status: Expired due to Fees

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Bangalore, Srinivas, Johnston, Michael
• Primary Examiner(s): Hudspeth, David R
• Assistant Examiner(s): Spooner, Lamont M
• Application Number:
  US10/325,292
• Publication Number:
  US 20040122674A1
• Time in Patent Office: 2,980 Days
• Field of Search: 704/9, 704/1, 704/231, 704/251, 704/257, 704/275, 707 2- 5, 715/715, 715/256, 345/619, 345/629, 709/228
• US Class Current: 704/231
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A system and method of presenting widgets to a user during a multi-modal interactive dialog between a user and a computer is presented. The system controls the multi-modal dialog; and when user input would help to clarify or speed up the presentation of requested information, the system presents a temporary widget to the user to elicit the user input in this regard. The system presents the widget on a display screen at a position that will not interfere with the dialog. Various types of widgets are available, such as button widgets, sliders and confirmation widgets, depending on the type of information that the system requires.

Claims

1. In a multi-modal dialog system, a method of providing widgets to a user, comprising, after first user input received in a combination of a first mode and a second mode in a map-based application, and where further user input will clarify the first user input during a multi-modal dialog:

maintaining a current display screen context by only presenting additional data on a display in response to the first user input until further user input is received via interaction with the additional data that clarifies the further user input;

maintaining a current dialog context;

presenting speech to the user requesting the further user input to clarify the first user input; and

presenting the slider widget in a corner of the display screen to elicit the first user input as directed by the presented speech, wherein the further user input is received via the widget in a non-speech mode and provides distance range data that is shown on the display screen as the slider widget is adjusted by the further user input.

2. The method of claim 1, wherein the user may manipulate the widget using a stylus, mouse, or touch.

3. The method of claim 1, wherein the widget appears at a location on the display screen that does not interfere with the multi-modal dialog.

4. The method of claim 3, wherein the multi-modal dialog system dynamically selects the location of the widget according to a multi-modal dialog context such that the widget does not cover relevant information on the display screen.

5. The method of claim 1, wherein the widget comprises a list of selectable buttons.

6. The method of claim 1, wherein the widget comprises arrow buttons.

7. A method of temporarily providing one of a plurality of widgets to a user in the course of a multi-modal dialog with a computer device in a map-based application, the method comprising:

after first user input received in a combination of a first mode and a second mode, determining whether further user input would be advantageous before the computer device presents information to the user;

if further user input would be advantageous, selecting a widget from a plurality of widgets to yield a selected widget, the selected widget being associated with the further user input;

maintaining a current display screen context by only presenting additional data on a display in response to the first user input until further user input is received via user interaction with the additional data that clarifies the further user input;

maintaining a current dialog context;

presenting speech to the user requesting the further user input to clarify the first user input; and

presenting the selected widget in a corner of the display for receiving the further user input as directed by the presented speech, wherein the further user input is received in a non-speech mode and provides distance range data that is shown on the display screen as the selected widget is adjusted by the further user input.

8. The method of claim 7, wherein the plurality of widgets comprises at least button widgets.

9. The method of claim 7, wherein the computer device dynamically generates the content of the selected widget according to the required further user input.

10. The method of claim 7, further comprising, when the user provides the further input, removing the selected widget from the display.

11. A multi-modal dialog system presenting a map-based application comprising:

a multi-modal interface module that receives multi-modal input from a user received in a combination of a first mode and a second mode and provides multi-modal information to the user;

a module that maintains a current dialog screen context by only presenting additional data on a display in response to the multimodal input until further user input is received via user interaction with the additional data that clarifies the multimodal input; and

a widget control module that dynamically presents the slider widget on the display screen to facilitate a multi-modal dialog between the user and the multi-modal dialog system, the slider widget receiving further user input in a third mode, the further user input clarifying the first user input and providing distance range data that is shown on the display screen as the slider widget is adjusted by the further user input.

12. The multi-modal dialog system of claim 11, wherein the widget control module selects the slider widget from a plurality of widgets.

13. The multi-modal dialog system of claim 12, wherein the plurality of widgets comprises at least a button widget.

14. The multi-modal dialog system of claim 11, wherein the widget control module only presents slider widgets in the course of the multi-modal dialog when the multi-modal dialog system receives first user input and further user input could refine the first user input.

15. The multi-modal dialog system of claim 11, wherein the user can provide user input via the widget by stylus, mouse control, or touch.

16. The multi-modal dialog system of claim 11, wherein the user can provide user input in response to the slider widget via speech rather than interacting with the widget via a non-speech mode.

17. The multi-modal dialog system of claim 11, wherein the widget module further presents slider widgets on the display screen at a location that does not interfere with the presentation of multi-modal information.

18. The multi-modal dialog system of claim 11, wherein the slider widgets are transparent.

19. The multi-modal dialog system of claim 11, wherein the widget module dynamically selects a location of the slider widget according to content on the display screen.

20. The multi-modal dialog system of claim 11, wherein the user can control the appearance of the slider widgets during the multi-modal dialog.