Method and system for interpreting verbal inputs in multimodal dialog system

• US 7,584,099 B2
• Filed: 04/06/2005
• Issued: 09/01/2009
• Est. Priority Date: 04/06/2005
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Cheng, Yan M., Bliss, Harry M., Ma, Changxue C.
• Primary Examiner(s): Vo; Huyen X.
• Application Number:
  US11/100,185
• Publication Number:
  US 20060229862A1
• Time in Patent Office: 1,609 Days
• Field of Search: 704/240, 704/257, 704/231, 704/235, 704/239, 704/238, 704/242
• US Class Current: 704/240
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning

Abstract

A method, a system and a computer program product for interpreting a verbal input in a multimodal dialog system are provided. The method includes assigning (302) a confidence value to at least one word generated by a verbal recognition component. The method further includes generating (304) a semantic unit confidence score for the verbal input. The generation of a semantic unit confidence score is based on the confidence value of at least one word and at least one semantic confidence operator.

Claims

1. A method for interpreting a verbal input in a multimodal dialog system, the multimodal system being an apparatus that comprises input and output devices and a processing system, the method comprising:

receiving the verbal input at an input device of the multi modal dialog system,

generating a word sequence from the verbal input by a verbal recognition component of the processing system using a recognition grammar, the word sequence comprising at least one word that is semantically tagged;

assigning by a value assigner component of the processing system a confidence value to each of the at least one word generated by the verbal recognition component, based on the word'"'"'s semantic importance;

generating by a score generator of the processing system a semantic unit confidence score for the word sequence, the generation of the semantic unit confidence score being based on the confidence value of each of the at least one word and at least one semantic confidence operator; and

generating an output by a human interface output device based on the semantic unit confidence score.

2. The method for interpreting a verbal input in accordance with claim 1 wherein the generation of the semantic unit confidence score is based on a semantic tag of a recognition grammar.

3. The method for interpreting a verbal input in accordance with claim 2 wherein the semantic tag comprises the at least one semantic confidence operator, the at least one word, and a confidence value associated with each of the at least one word.

4. The method for interpreting a verbal input in accordance with claim 1 wherein the at least one semantic confidence operator is selected from a group consisting of average, selection, maximum confidence and minimum confidence.

5. The method for interpreting a verbal input in accordance with claim 1 wherein the multimodal dialog system is a modular-portable dialog (MPD) system.

6. The method for interpreting a verbal input in accordance with claim 1 further comprising:

initiating an error dialog when the semantic unit confidence score is less than a threshold value.

7. The method for interpreting a verbal input in accordance with claim 1, wherein a semantic group confidence score is generated from the semantic unit confidence score and at least one other semantic unit confidence score during a turn of the dialog.

8. The method for interpreting a verbal input in accordance with claim 7 wherein one of a semantic sentence and a semantic turn confidence score is generated from more than one of:

the semantic unit confidence score;

the at least one other semantic unit confidence score; and

a semantic sentence score.

9. A multimodal dialog system for interpreting a verbal input, the multimodal dialog system comprising:

an input device that receives the verbal input;

an input processor that comprises;

a verbal recognition component that generates a word sequence comprising at least one word from the verbal input using on a recognition grammar, the word sequence comprising at least one word that is semantically tagged,

a value assigner component, that assigns a confidence value to each of the at least one word generated by the verbal recognition component, based on the word'"'"'s semantic importance, and

a score generator that generates a semantic unit confidence score for the verbal input, the generation of the semantic unit confidence score being based on the confidence value of the at least one word and at least one semantic confidence operator; and

a human output device that generates a human interface output based on the semantic unit confidence score.

10. The multimodal dialog system in accordance with claim 9 wherein the generation of the semantic unit confidence score is based on a semantic tag of a recognition grammar.

11. The multimodal dialog system in accordance with claim 10 wherein the semantic tag comprises the at least one semantic confidence operator, the at least one word, and a confidence value associated with each of the at least one word.

12. The multimodal dialog system in accordance with claim 9 wherein the at least one semantic confidence operator is selected from a group consisting of average, selection, maximum confidence and minimum confidence.

13. The multimodal dialog system in accordance with claim 9 wherein an error dialog is initiated when the semantic unit confidence score is less than a threshold value.

14. The multimodal dialog system in accordance with claim 9 wherein a semantic group confidence score is generated from the semantic unit confidence score and at least one other semantic unit confidence score during a turn of the dialog.

15. The multimodal dialog system in accordance with claim 14 wherein one of a semantic sentence and a semantic turn confidence score is generated from more than one of:

the semantic unit confidence score;

the at least one other semantic unit confidence score; and

a semantic sentence score.