Method and system for intelligent prompt control in a multimodal software application

• US 20050010418A1
• Filed: 07/10/2003
• Published: 01/13/2005
• Est. Priority Date: 07/10/2003
• Status: Abandoned Application

• Current Assignee: Vocollect, Inc. (Honeywell International Inc.)
• Original Assignee: Vocollect, Inc. (Honeywell International Inc.)
• Inventors: Sweeney, Lawrence R., Eusterman, Timothy Joseph, McNair, Arthur Eugene
• Application Number:
  US10/617,593
• Publication Number:
  US 20050010418A1
• Time in Patent Office: Days
• US Class Current: 704/275
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

Dialog manager and methods for integrating multi-modal data capture device inputs or speech recognition inputs with speech output capabilities. A work flow description is extracted from objects in a graphical user interface and a multi-modal user interface is defined. A dialog engine synchronizes the flow of information, in accordance with the work flow description, between input/output devices and an application. The prompts for inputting data, which are output via a plurality of peripheral devices, are controlled in an intelligent manner by the dialog engine based on the input state of the peripheral devices. Functionality such as barge-in, prompt-holdoff, priority prompts, and talk-ahead is provided.

Claims

1. A system for executing a multimodal software application, comprising:

the multimodal software application, wherein said multimodal software application is configured to receive first data input from a first set of peripheral devices and output second data to a second set of peripheral devices;

a dialog engine in communication with the multimodal software application, wherein said dialog engine is configured to execute a workflow description received from the multimodal software application and provide the first data to the multimodal software application;

said dialog engine further configured to control outputting of a prompt from the workflow description based on an input state of the first set of peripheral devices; and

a respective interface component associated with each peripheral device within said first and second sets;

wherein each interface component is configured to provide the second data, if any, to the associated peripheral device and receive the first data, if any, from the associated peripheral device.

2. The system according to claim 1, wherein said control includes interrupting the prompt if the first data is received while the prompt is being output.

3. The system according to claim 1, wherein said control includes delaying outputting of the prompt if one of the first set of peripheral devices is receiving the first data.

4. The system according to claim 1, wherein said control includes determining that the first data relates to the prompt and a subsequent prompt, and associating a portion of the first data with the prompt and associating another portion of the first data with the subsequent prompt.

5. The system according to claim 4, wherein said control further includes avoiding the output of the subsequent prompt.

6. The system according to claim 2, wherein said control further includes preventing interrupting and terminating the prompt if the prompt is designated as non-interruptible.

7. The system according to claim 1, wherein the first set of peripheral devices includes one or more of a voice recognition system, a radio-frequency identifier scanner, a bar code scanner, a touch screen, a keypad, and a computer.

8. The system according to claim 1, wherein the second set of peripheral devices includes one or more of a voice synthesis system, a display screen and a computer.

9. A method for executing a multimodal application, comprising the steps of:

executing a workflow description received from the multimodal application, said workflow description including a plurality of workflow objects;

outputting a prompt of a first workflow object via a plurality of peripheral devices, said prompt related to the multimodal application; and

controlling the outputting of the prompt according to an input state of the plurality of peripheral devices.

10. The method according to claim 9, wherein the prompt relates to a visual control of a GUI screen of the multimodal application.

11. The method according to claim 9, wherein the step of controlling includes the steps of:

receiving data before said step of outputting completes; and

in response to receiving the data, terminating the outputting step whereby any remaining portion of the prompt is not output.

12. The method according to claim 11, wherein:

the step of outputting includes outputting an audio prompt; and

the step of receiving includes receiving voice data from a speech recognition system.

13. The method according to claim 11, wherein the data is received from one of the plurality of peripheral devices.

14. The method according to claim 11 further comprising the steps of:

determining if the prompt has been designated as non-interruptible; and

preventing terminating of the prompt.

15. The method according to claim 11, further comprising the steps of:

performing the step of terminating if the data is received from a predetermined peripheral device; and

omitting the step of terminating if the input is received from other than the predetermined device.

16. The method according to claim 9, wherein the step of controlling includes the steps of:

receiving data, in response to the prompt, related to the prompt and a second workflow object; and

associating a portion of the data with the first workflow object and another portion of the data with the second workflow object.

17. The method according to claim 16, further comprising the step of:

preventing output of a subsequent prompt related to the second workflow object.

18. The method according to claim 16, wherein the data relates to the first workflow object and a plurality of other workflow objects.

19. The method according to claim 9, wherein the step of controlling includes the steps of:

receiving data at one of the plurality of peripheral devices; and

delaying the step of outputting the prompt until the data is no longer being received.

20. The method according to claim 19, wherein the step of delaying includes the steps of:

delaying outputting the prompt to the one peripheral devices; and

permitting outputting the prompt without delay to another of the plurality of peripheral devices.

21. The method according to claim 19, further comprising the steps of:

determining if the data relates to the prompt; and

omitting outputting of the prompt if the data relates to the prompt.

22. A computer-readable medium bearing instructions for executing a multimodal application, said instructions being arranged, upon execution thereof, to cause one or more processors to perform the steps of:

executing a workflow description received from the multimodal application, said workflow description including a plurality of workflow objects;

outputting a prompt of a first workflow object via a plurality of peripheral devices, said prompt related to a visual control of a GUI screen of the multimodal application; and

controlling the outputting of the prompt according to an input state of the plurality of peripheral devices.

23. The computer-readable medium according to claim 22, wherein the instructions are further arranged, upon execution thereof, to cause the one or more processors to perform the steps of:

receiving data before said step of outputting completes; and

in response to receiving the data, terminating the outputting step whereby any remaining portion of the prompt is not output.

24. The computer-readable medium according to claim 22, wherein the instructions are further arranged, upon execution thereof, to cause the one or more processors to perform the steps of:

receiving data, in response to the prompt, related to the prompt and a second workflow object; and

associating a portion of the data with the first workflow object and another portion of the data with the second workflow object.

25. The computer-readable medium according to claim 22, wherein the instructions are further arranged, upon execution thereof, to cause the one or more processors to perform the steps of:

receiving data at one of the plurality of peripheral devices; and

delaying the step of outputting the prompt until the data is no longer being received.