System and method of context-sensitive help for multi-modal dialog systems

• US 20040006475A1
• Filed: 12/19/2002
• Published: 01/08/2004
• Est. Priority Date: 07/05/2002
• Status: Active Grant

• Current Assignee: Interactions, LLC
• Original Assignee: AT&T Corporation (AT&T, Inc.)
• Inventors: Johnston, Michael, Ehlen, Patrick, Hastie, Helen
• Granted Patent: US 7,177,815 B2
• Time in Patent Office: Days
• US Class Current: 704/270.1
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/221 Announcement of recognition results
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A method of presenting a multi-modal help dialog move to a user in a multi-modal dialog system is disclosed. The method comprises presenting an audio portion of the multi-modal help dialog move that explains available ways of user inquiry and presenting a corresponding graphical action performed on a user interface associated with the audio portion. The multi-modal help dialog move is context-sensitive and uses current display information and dialog contextual information to present a multi-modal help move that is currently related to the user. A user request or a problematic dialog detection module may trigger the multi-modal help move.

Claims

1. In a multi-modal dialog system, a method of presenting a multi-modal help dialog move to a user, the method comprising:

presenting an audio portion of the multi-modal help dialog move that explains available ways of user inquiry; and

presenting corresponding graphical action on a user interface coordinated with the audio portion.

2. The method of claim 1, wherein presenting the audio portion and presenting the graphical action occur in response to a user inquiry.

3. The method of claim 1, wherein presenting the audio portion and presenting the graphical action occur in response to a problematic dialog situation.

4. The method of claim 1, wherein the presented audio portion and presented graphical action relate to what is currently being displayed to the user.

5. The method of claim 1, wherein the presented graphical action further comprises at least one graphical button corresponding to available choices to the user.

6. The method of claim 5, wherein the at least one graphical button relates to different kinds of help available to the user.

7. In a multi-modal dialog system, a method of presenting a multi-modal help dialog move to a user, comprising:

analyzing a current dialog state with the user, including a visual context, to determine whether presenting a multi-modal help dialog move to the user is appropriate;

if the multi-modal dialog system determines that it is appropriate to present a multi-modal help dialog move;

presenting the multi-modal help dialog move having an audio portion and a graphical portion that utilize the visual context of the current dialog state with the user.

8. The method of claim 7, further comprising:

if the multi-modal dialog system determines that it is not appropriate to present a multi-modal help dialog move;

waiting for a user request to present a multi-modal help dialog move.

9. The method of claim 7, wherein the presented graphical portion further comprises at least one graphical button corresponding to available choices to the user.

10. The method of claim 9, where the at least one graphical button relates to different kinds of help available to the user.

11. In a multi-modal dialog system, a method of presenting a multi-modal help dialog move to a user, comprising:

upon a user request or triggered by a problematic dialog situation detector, presenting the multi-modal help dialog move having an audio portion and a graphical portion that utilize a visual context of the current dialog state with the user, wherein the multi-modal help dialog move provides information to the user regarding how to interact with the system in a multi-modal fashion.

12. The method of claim 11, wherein the presented graphical portion further comprises at least one graphical button corresponding to available choices to the user.

13. The method of claim 12, where the at least one graphical button relates to different kinds of help available to the user.

14. In a multi-modal dialog system using a map display having an entity of interest thereon, a method of presenting a multi-modal help dialog move to a user, comprising:

when an entity of interest is currently being displayed to the user;

presenting a multi-modal help dialog move having an audio portion and a graphical portion, the audio portion stating a plurality of types of information available to the user associated with the entity of interest, and the graphical portion corresponding to the audio portion.

15. The method of claim 14, where the audio portion further comprises instructions to the user regarding how to multi-modally obtain information regarding the entity of interest.

16. The method of claim 15, wherein while the audio portion is being presented to the user explaining how to multi-modally obtain information regarding the entity of interest, the graphical portion provides an example graphical input that the user can make to obtain the information about the entity of interest.

17. A method of assisting a user to interact with a system having a multi-modal interface, the method comprising:

analyzing a multi-modal dialog for problematic situations; and

upon determining that help is needed, presenting to the user a multi-modal dialog help move having an audio portion and a graphical portion, the multi-modal dialog help move audibly explaining the user available multi-modal actions the user can take to obtain information while providing corresponding graphics demonstrating the kind of graphic input the user can make to obtain the information.

18. The method of claim 17, wherein the system comprises a problematic dialog situation detector to detect problematic situations.

19. The method of claim 17, wherein the presented multi-modal dialog help move utilizes current graphical information being presented to the user in demonstrating the kind of graphic input the user can make to obtain the information.

20. In a multi-modal dialog system using a map display having a plurality of entities of interest thereon, a method of presenting a multi-modal help dialog move to a user, comprising:

selecting an entity of interest from the plurality of entities of interest currently displayed to the user; and

presenting a multi-modal help dialog move having an audio portion and a graphical portion, the audio portion stating a plurality of types of information available to the user associated with the selected entity of interest, and the graphical portion being associated with the audio portion.

21. In a multi-modal dialog system, a method of presenting a multi-modal help dialog move to a user, the method comprising:

presenting text associated with the help dialog move; and

presenting corresponding graphical action on a user interface coordinated with the text, wherein the presented text and presented graphical action are context-sensitive.

22. The method of claim 21, wherein presenting the text and presenting the graphical action occur in response to a user inquiry.

23. The method of claim 21, wherein presenting the text and presenting the graphical action occur in response to a problematic dialog situation.

24. The method of claim 21, further comprising:

presenting synthetic speech associated with the presented text.

25. The method of claim 24, wherein a text-related graphical action to graphically synchronize the synthetic speech with the text.

26. The method of claim 25, wherein the text related graphical action is a bouncing ball.

27. The method of claim 25, wherein the text-related graphical action is highlighting a word in the text currently being spoken by the synthetic speech.