System and method of handling problematic input during context-sensitive help for multi-modal dialog systems

• US 7,451,088 B1
• Filed: 01/03/2007
• Issued: 11/11/2008
• Est. Priority Date: 07/05/2002
• Status: Expired due to Term

• Current Assignee: Interactions, LLC
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Ehlen, Patrick, Hastie, Helen, Johnston, Michael
• Primary Examiner(s): Hudspeth, David R.
• Assistant Examiner(s): Ng, Eunice
• Application Number:
  US11/619,325
• Time in Patent Office: 678 Days
• Field of Search: None
• US Class Current: 704/270.1
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/221 Announcement of recognition results
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A method of presenting a multi-modal help dialog move to a user in a multi-modal dialog system is disclosed. The method comprises presenting an audio portion of the multi-modal help dialog move that explains available ways of user inquiry and presenting a corresponding graphical action performed on a user interface associated with the audio portion. The multi-modal help dialog move is context-sensitive and uses current display information and dialog contextual information to present a multi-modal help move that is currently related to the user. A user request or a problematic dialog detection module may trigger the multi-modal help move.

Claims

1. In a multi-modal dialog system, a method of presenting a multi-modal help dialog move to a user in a problematic dialog situation, the method comprising:

receiving a plurality of utterances that do not meet a successful recognition threshold;

in response to the received plurality of utterances, presenting an audio portion of a multi-modal help dialog move that explains available non-audio ways of user inquiry; and

presenting a corresponding graphical action performed on a user interface associated with the audio portion.

2. The method of claim 1, wherein the presented audio portion and presented graphical action relate to what is currently being displayed to the user.

3. The method of claim 1, wherein the presented graphical action further comprises at least one graphical button corresponding to choices available to the user.

4. The method of claim 3, wherein the at least one graphical button relates to different kinds of help available to the user.

5. The method of claim 1, wherein the number of the plurality of user utterances that does not meet the successful recognition threshold is adjustable based on user experience.

6. The method of claim 1, wherein the number of the plurality of user utterances that does not meet the successful recognition threshold is adjustable based on automatic speech recognition performance.

7. The method of claim 1, further comprising:

receiving at least one graphical user input that does not meet a graphical user input recognition threshold.

8. The method of claim 7, wherein the presented audio portion and presented graphical action relate to what is currently being displayed to the user.

9. The method of claim 7, wherein the presented graphical action further comprises at least one graphical button corresponding to choices available to the user.

10. The method of claim 9, wherein the at least one graphical button relates to different kinds of help available to the user.

11. The method of claim 1, wherein the number of plurality of utterances is set according to user experience and automatic speech recognition parameters.

17. The system of claim 11, wherein the number of the plurality of user utterances that does not meet the successful recognition threshold is adjustable based on automatic speech recognition performance.

12. A multi-modal dialog system for presenting a multi-modal help dialog move to a user, in a problematic dialog situation, the system comprising:

a module receiving a plurality of utterances that do not meet a successful recognition threshold;

in response to the received plurality of utterances, a module presenting an audio portion of a multi-modal help dialog move that explains available non-audio ways of user inquiry; and

a module presenting a corresponding graphical action performed on a user interface associated with the audio portion.

13. The system of claim 12, wherein the presented audio portion and presented graphical action relate to what is currently being displayed to the user.

14. The system of claim 12, wherein the presented graphical action further comprises at least one graphical button corresponding to choices available to the user.

15. The system of claim 14, wherein the at least one graphical button relates to different kinds of help available to the user.

16. The system of claim 12, wherein the number of the plurality of user utterances that does not meet the successful recognition threshold is adjustable based on user experience.

18. A computer readable medium storing a computer program for controlling a computing device to present a multi-modal help dialog move to a user in a problematic dialog situation, the computer program having instructions comprising:

receiving a plurality of utterances that do not meet a successful recognition threshold;

in response to the received plurality of utterances, presenting an audio portion of a multi-modal help dialog move that explains available non-audio ways of user inquiry; and

presenting a corresponding graphical action performed on a user interface associated with the audio portion.

19. The computer readable medium of claim 18, wherein the presented audio portion and presented graphical action relate to what is currently being displayed to the user.

20. The computer readable medium of claim 18, wherein the presented graphical action further comprises at least one graphical button corresponding to choices available to the user.