Automatic interpreting system including a system for recognizing errors

• US 7,219,050 B2
• Filed: 09/07/2001
• Issued: 05/15/2007
• Est. Priority Date: 09/11/2000
• Status: Expired due to Fees

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Ishikawa, Kai, Ando, Shinichi, Okumura, Akitoshi
• Primary Examiner(s): CHAWAN, VIJAY B
• Application Number:
  US09/947,438
• Publication Number:
  US 20020032561A1
• Time in Patent Office: 2,076 Days
• Field of Search: 704/3, 704/9, 704/2, 704/257
• US Class Current: 704/3
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding

Abstract

An automatic interpreting system, having at least an inputting unit for inputting utterances, an interpreting unit for recognizing the input utterance and interpreting the input utterance into a specified language, and an outputting unit for outputting interpretation results, includes a conversation history managing unit for determining the level of interpretation reliability calculated based on the interpretation result, having the user input their level of interpretation understanding, and recording and storing this data as interpretation history information, and a conversation status determination processing unit for, if the interpretation of an utterance is not understandable to the receiving party, determining the conversation status by selecting from categories pre-determined based on the levels of interpretation reliability and interpretation understanding for the previous utterance that are stored in the conversation history managing unit, and a recommended action presenting part for presenting to the speaker, as recommended actions, pre-determined actions for the conversation status.

Claims

1. An automatic interpreting system comprising:

an inputting device that receives an input utterance;

an interpreting device that recognizes the input utterance and interprets the input utterance into a specified language;

an outputting device that outputs interpretation results;

a conversation history managing unit that determines the validity of the interpretation result of the input utterance to obtain interpretation reliability,

allows the receiving-side speaker to input as the level of interpretation understanding how well said receiving-speaker can grasp the meaning of the interpretation result, and

records and stores, as interpretation history information, said level of interpretation reliability and said level of interpretation understanding for each said utterance; and

a conversation status determination processing unit that, if the interpretation of an utterance is not understandable to the receiving party, determines the conversation status, based on said interpretation reliability and said interpretation understanding, for said utterance and a previous utterance, which are stored in said conversation history managing unit.

2. The automatic interpreting system as set forth in claim 1, comprising:

recommended action determination processing unit that outputs to the speaker, as recommended actions, pre-determined actions for the conversation status determined by said conversation status determination processing unit.

3. The automatic interpreting system as set forth in claim 1, wherein

if the level of interpretation understanding for an utterance is “

not understandable,”

then the current conversation status is determined, i.e., whether an error has occurred in the interpretation result of said utterance or a misunderstanding has occurred in the interpretation result of the previous utterance, by putting the combinations of said interpretation reliability and said interpretation understanding for said utterance and for the previous utterance in contrast with the pre-determined categories.

4. The automatic interpreting system as set forth in claim 1, comprising

recommended action determination processing unit that outputs to the speaker, as recommended actions, pre-determined actions for the conversation status determined by said conversation status determination processing unit, wherein

if the level of interpretation understanding for an utterance is “

not understandable,”

then the current conversation status is determined, i.e., whether an error has occurred in the interpretation result of said utterance or a misunderstanding has occurred in the interpretation result of the previous utterance, by putting the combinations of said interpretation reliability and said interpretation understanding for said utterance and for the previous utterance in contrast with the pre-determined categories.

5. The automatic interpreting system as set forth in claim 3, wherein

if said interpretation reliability is categorized into two values, high and low, and said interpretation understanding is categorized into two values, clear and unclear, and if said interpretation understanding for an utterance is “

unclear,”

if the value of said interpretation reliability for said utterance is “

high” and

the values of said interpretation reliability and said interpretation understanding for the previous utterance are “

low” and

“

clear,”

respectively, the conversation status determination processing unit determines that a misunderstanding has occurred in the interpretation result of the previous utterance and sets the conversation status, and

if the value of said interpretation reliability for said utterance is “

high” and

the values of said interpretation reliability and said interpretation understanding for the previous utterance are “

high” and

“

clear,”

respectively, or if the value of said interpretation reliability for said utterance is “

low” and

the value of said interpretation understanding for the previous utterance is “

clear,”

the conversation status determination processing unit determines that an error has occurred in the interpretation result of said utterance and sets the conversation status.

6. The automatic interpreting system as set forth in claim 1, wherein

actions to be presented by said recommended action determination processing unit are selected from options, including requesting re-utterance, re-uttering, requesting check of recognition, looking up dictionary, and selecting next translation candidate.

7. The automatic interpreting system as set forth in claim 6, wherein

in conversation between a speaker using the first language and another speaker using the second language,

if the values of said interpretation reliability and said interpretation understanding for the current utterance in the first language are “

high” and

“

unclear,”

respectively, and the values of said interpretation reliability and said interpretation understanding for the previous utterance in the second language “

low” and

“

clear,”

respectively, “

request re-utterance from second speaker”

is presented by said recommended action determination processing unit,

if the values of said interpretation reliability and said interpretation understanding for the current utterance in the first language are “

low” and

“

unclear,”

respectively, and the value of said interpretation reliability for the previous utterance in the second language is “

high,”

or if the values of said interpretation reliability and said interpretation understanding for the current utterance in the first language are “

low” and

“

unclear,”

respectively, and the values of said interpretation reliability and said interpretation understanding for the previous utterance in the second language are “

high” and

“

clear,”

respectively, “

first speaker re-utters” and

“

first speaker checks recognition”

are presented by said recommended action determination processing unit,

if the values of said interpretation reliability and said interpretation understanding for the current utterance in the second language are “

high” and

“

unclear,”

respectively, and the values of said interpretation reliability and said interpretation understanding for the previous utterance in the first language “

low” and

“

clear,”

respectively, “

first speaker re-utters”

is presented by said recommended action determination processing unit, and

if the values of said interpretation reliability and said interpretation understanding for the current utterance in the second language are “

low” and

“

unclear,”

respectively, and the value of said interpretation reliability for the previous utterance in the first language is “

high,”

or if the values of said interpretation reliability and said interpretation understanding for the current utterance in the second language are “

high” and

“

unclear,”

respectively, and the values of said interpretation reliability and said interpretation understanding for the previous utterance in the first language are “

high” and

“

clear,”

respectively, “

second speaker re-utters,”

“

second speaker checks recognition,”

“

first speaker looks up dictionary”

, and “

first speaker selects next translation candidate”

are presented by said recommended action determination processing unit.

8. The automatic interpreting system as set forth in claim 1, comprising:

recommended action determination processing unit that outputs to the speaker, as recommended actions, pre-determined actions for the conversation status determined by said conversation status determination processing unit, wherein

actions to be presented by said recommended action determination processing unit are selected from options, including requesting re-utterance, re-uttering, requesting check of recognition, looking up dictionary, and selecting next translation candidate.

9. The automatic interpreting system as set forth in claim 1, wherein

if the level of interpretation understanding for an utterance is “

not understandable,”

then the current conversation status is determined, i.e., whether an error has occurred in the interpretation result of said utterance or a misunderstanding has occurred in the interpretation result of the previous utterance, by putting the combinations of said interpretation reliability and said interpretation understanding for said utterance and for the previous utterance in contrast with the pre-determined categories, and

actions to be presented by said recommended action determination processing unit are selected from options, including requesting re-utterance, re-uttering, requesting check of recognition, looking up dictionary, and selecting next translation candidate.

10. The automatic interpreting system as set forth in claim 1, wherein

assuming that said interpretation reliability is categorized into two values, high and low, and said interpretation understanding is categorized into two values, clear and unclear, and

if said interpretation understanding for an utterance is “

unclear,”

if the value of said interpretation reliability for said utterance is “

high” and

said interpretation reliability and said interpretation understanding for the previous utterance are “

low” and

“

clear,”

respectively, the conversation status determination processing unit determines that a misunderstanding has occurred in the interpretation result of the previous utterance and sets the conversation status, and

if the value of said interpretation reliability for said utterance is “

high” and

the values of said interpretation reliability and said interpretation understanding for the previous utterance are “

high” and

“

clear,”

respectively, or if the value of said interpretation reliability for said utterance is “

low” and

the value of said interpretation understanding for the previous utterance is “

clear,”

the conversation status determination processing unit determines that an error has occurred in the interpretation result of said utterance and sets the conversation status, and

actions to be presented by said recommended action determination processing unit are selected from options, including requesting re-utterance, re-uttering, requesting check of recognition, looking up dictionary, and selecting next translation candidate.

11. An automatic interpreting method for recognizing an input utterance, interpreting the input utterance into a specified language, and outputting an interpretation result, comprising the steps of:

determining the validity of the interpretation result of said input utterance to obtain the level of interpretation reliability, and allowing the receiving-side speaker to input as the level of interpretation understanding how well she said receiving-side speaker can grasp the meaning of the interpretation result,

recording and storing, as interpretation history information, said level of interpretation reliability and said level of interpretation understanding for each said utterance, and

if the interpretation of an utterance is not understandable to the receiving party, determining the conversation status;

based on said interpretation reliability and said interpretation understandings for said utterance and a previous utterance, which are stored as the interpretation history information.

12. The automatic interpreting method as set forth in claim 11, wherein pre-determined actions according to said conversation status are presented to the speaker as recommended actions.

13. The automatic interpreting method as set forth in claim 11, wherein

if the level of interpretation understanding for said utterance is “

not understandable,”

then the current conversation status is determined, i.e., whether an error has occurred in the interpretation result of said utterance or a misunderstanding has occurred in the interpretation result of the previous utterance, by putting the combinations of said interpretation reliability and said interpretation understanding for said utterance and for the previous utterance in contrast with the pre-determined categories.

14. The automatic interpreting method as set forth in claim 11, wherein

pre-determined actions for said conversation status are presented to the speaker as recommended actions, and

if the level of interpretation understanding for an utterance is “

not understandable,”

then the current conversation status is determined, i.e., whether an error has occurred in the interpretation result of said utterance or a misunderstanding has occurred in the interpretation result of the previous utterance, by putting the combinations of said interpretation reliability and said interpretation understanding for said utterance and for the previous utterance in contrast with the pre-determined categories.

15. The automatic interpreting method as set forth in claim 11, wherein

if the level of interpretation understanding for an utterance is “

not understandable,”

then the current conversation status is determined, i.e., whether an error has occurred in the interpretation result of said utterance or a misunderstanding has occurred in the interpretation result of the previous utterance, by putting the combinations of said interpretation reliability and said interpretation understanding for said utterance and for the previous utterance in contrast with the pre-determined categories,

assuming that said interpretation reliability is categorized into two values, high and low, and said interpretation understanding is categorized into two values, clear and unclear, and

if said interpretation understanding for an utterance is “

unclear,”

if the value of said interpretation reliability for said utterance is “

high” and

said interpretation reliability and said interpretation understanding for the previous utterance are “

low” and

“

clear,”

respectively, the conversation status is determined to be a misunderstanding that has occurred in the interpretation result of the previous utterance and the conversation status is then set, and

if the value of said interpretation reliability for said utterance is “

high” and

the values of said interpretation reliability and said interpretation understanding for the previous utterance are “

high” and

“

clear,”

respectively, or if the value of said interpretation reliability for said utterance is “

low” and

the value of said interpretation understanding for the previous utterance is “

clear,”

the conversation status is determined to be an error that has occurred in the interpretation result of said utterance and the conversation status is then set.

16. The automatic interpreting method as set forth in claim 11, wherein

pre-determined actions for said conversation status are presented to the speaker as recommended actions, and

actions to be presented as recommended actions are selected from options, including requesting re-utterance, re-uttering, requesting check of recognition, looking up dictionary, and selecting next translation candidate.

17. The automatic interpreting method as set forth in claim 16, wherein

in conversation between a speaker using the first language and another speaker using the second language,

if the values of said interpretation reliability and said interpretation understanding for the current utterance in the first language are “

high” and

“

unclear,”

respectively, and the values of said interpretation reliability and said interpretation understanding for the previous utterance in the second language “

low” and

“

clear,”

respectively, “

request re-utterance from second speaker”

is presented as the recommended action,

if the values of said interpretation reliability and said interpretation understanding for the current utterance in the first language are “

low” and

“

unclear,”

respectively, and the value of said interpretation reliability for the previous utterance in the second language is “

high,”

or if the values of said interpretation reliability and said interpretation understanding for the current utterance in the first language are “

low” and

“

unclear,”

respectively, and the values of said interpretation reliability and said interpretation understanding for the previous utterance in the second language are “

high” and

“

clear,”

respectively, “

first speaker re-utters” and

“

first speaker checks recognition”

are presented as the recommended actions,

if the values of said interpretation reliability and said interpretation understanding for the current utterance in the second language are “

high” and

“

unclear,”

respectively, and the values of said interpretation reliability and said interpretation understanding for the previous utterance in the first language “

low” and

“

clear,”

respectively, “

first speaker re-utters”

is presented as the recommended action, and

if the values of said interpretation reliability and said interpretation understanding for the current utterance in the second language are “

low” and

“

unclear,”

respectively, and the value of said interpretation reliability for the previous utterance in the first language is “

high,”

or if the values of said interpretation reliability and said interpretation understanding for the current utterance in the second language are “

high” and

“

unclear,”

respectively, and the values of said interpretation reliability and said interpretation understanding for the previous utterance in the first language are “

high” and

“

clear,”

respectively, “

second speaker re-utters,”

“

second speaker checks recognition,”

“

first speaker looks up dictionary”

, and “

first speaker selects next translation candidate”

are presented as the recommended actions.

18. The automatic interpreting method as set forth in claim 11, wherein

if the level of interpretation understanding for an utterance is “

not understandable,”

then the current conversation status is determined, i.e., whether an error has occurred in the interpretation result of said utterance or a misunderstanding has occurred in the interpretation result of the previous utterance, by putting the combinations of said interpretation reliability and said interpretation understanding for said utterance and for the previous utterance in contrast with the pre-determined categories, and

actions to be presented as recommended actions are selected from options, including requesting re-utterance, re-uttering, requesting check of recognition, looking up dictionary, and selecting next translation candidate.

19. The automatic interpreting method as set forth in claim 11, wherein

if the level of interpretation understanding for an utterance is “

not understandable,”

then the current conversation status is determined, i.e., whether an error has occurred in the interpretation result of said utterance or a misunderstanding has occurred in the interpretation result of the previous utterance, by putting the combinations of said interpretation reliability and said interpretation understanding for said utterance and for the previous utterance in contrast with the pre-determined categories,

assuming that said interpretation reliability is categorized into two values, high and low, and said interpretation understanding is categorized into two values, clear and unclear, and if said interpretation understanding for an utterance is “

unclear,”

if the value of said interpretation reliability for said utterance is “

high” and

said interpretation reliability and said interpretation understanding for the previous utterance are “

low” and

“

clear,”

respectively, the conversation status is determined to be a misunderstanding that has occurred in the interpretation result of the previous utterance and the conversation status is then set,

if the value of said interpretation reliability for said utterance is “

high” and

the values of said interpretation reliability and said interpretation understanding for the previous utterance are “

high” and

“

clear,”

respectively, or if the value of said interpretation reliability for said utterance is “

low” and

the value of said interpretation understanding for the previous utterance is “

clear,”

the conversation status is determined to be an error that has occurred in the interpretation result of said utterance and the conversation status is then set, and

actions to be presented as the recommended actions are selected from options, including requesting re-utterance, re-uttering, requesting check of recognition, looking up dictionary, and selecting next translation candidate.

20. An automatic interpreting program for recognizing an input utterance, interpreting the input utterance into a specified language, and outputting an interpretation result, comprising the functions of:

determining the validity of the interpretation result of said input utterance to obtain the level of interpretation reliability, and allowing the receiving-side speaker to input as the level of interpretation understanding how well she said receiving-side speaker can grasp the meaning of the interpretation result,

recording and storing, as interpretation history information, said level of interpretation reliability and said level of interpretation understanding for each said utterance, and

if the interpretation of an utterance is not understandable to the receiving party, determining the conversation status, based on said interpretation reliability and said interpretation understanding, for said utterance and a previous utterance, which are stored as said interpretation history information.

21. The automatic interpreting program as set forth in claim 20, wherein

pre-determined actions according to said conversation status are presented to the speaker as recommended actions.

22. The automatic interpreting program as set forth in claim 20, wherein

if the level of interpretation understanding for said utterance is “

not understandable,”

then the current conversation status is determined, i.e., whether an error has occurred in the interpretation result of said utterance or a misunderstanding has occurred in the interpretation result of the previous utterance, by putting the combinations of said interpretation reliability and said interpretation understanding for said utterance and for the previous utterance in contrast with the pre-determined categories.

23. The automatic interpreting program as set forth in claim 20, wherein

pre-determined actions for said conversation status are presented to the speaker as recommended actions, and

if the level of interpretation understanding for an utterance is “

not understandable,”

then the current conversation status is determined, i.e., whether an error has occurred in the interpretation result of said utterance or a misunderstanding has occurred in the interpretation result of the previous utterance, by putting the combinations of said interpretation reliability and said interpretation understanding for said utterance and for the previous utterance in contrast with the pre-determined categories.

24. The automatic interpreting program as set forth in claim 20, wherein

if the level of interpretation understanding for an utterance is “

not understandable,”

then the current conversation status is determined, i.e., whether an error has occurred in the interpretation result of said utterance or a misunderstanding has occurred in the interpretation result of the previous utterance, by putting the combinations of said interpretation reliability and said interpretation understanding for said utterance and for the previous utterance in contrast with the pre-determined categories,

assuming that said interpretation reliability is categorized into two values, high and low, and said interpretation understanding is categorized into two values, clear and unclear, and if said interpretation understanding for an utterance is “

unclear,”

if the value of said interpretation reliability for said utterance is “

high” and

said interpretation reliability and said interpretation understanding for the previous utterance are “

low” and

“

clear,”

respectively, determining the conversation status to be a misunderstanding that has occurred in the interpretation result of the previous utterance and setting the conversation status, and

if the value of said interpretation reliability for said utterance is “

high” and

the values of said interpretation reliability and said interpretation understanding for the previous utterance are “

high” and

“

clear,”

respectively, or if the value of said interpretation reliability for said utterance is “

low” and

the value of said interpretation understanding for the previous utterance is “

clear,”

determining the conversation status to be an error that has occurred in the interpretation result of said utterance and setting the conversation status.

25. The automatic interpreting program as set forth in claim 20, wherein

pre-determined actions for said conversation status are presented to the speaker as recommended actions, and

actions to be presented as recommended actions are selected from options, including requesting re-utterance, re-uttering, requesting check of recognition, looking up dictionary, and selecting next translation candidate.

26. The automatic interpreting program as set forth in claim 25, wherein

in conversation between a speaker using the first language and another speaker using the second language,

if the values of said interpretation reliability and said interpretation understanding for the current utterance in the first language are “

high” and

“

unclear,”

respectively, and the values of said interpretation reliability and said interpretation understanding for the previous utterance in the second language “

low” and

“

clear,”

respectively, presenting “

request re-utterance from second speaker”

as the recommended action,

if the values of said interpretation reliability and said interpretation understanding for the current utterance in the first language are “

low” and

“

unclear,”

respectively, and the value of said interpretation reliability for the previous utterance in the second language is “

high,”

or if the values of said interpretation reliability and said interpretation understanding for the current utterance in the first language are “

low” and

“

unclear,”

respectively, and the values of said interpretation reliability and said interpretation understanding for the previous utterance in the second language are “

high” and

“

clear,”

respectively, presenting “

first speaker re-utters” and

“

first speaker checks recognition”

as the recommended actions,

if the values of said interpretation reliability and said interpretation understanding for the current utterance in the second language are “

high” and

“

unclear,”

respectively, and the values of said interpretation reliability and said interpretation understanding for the previous utterance in the first language “

low” and

“

clear,”

respectively, presenting “

first speaker re-utters”

as the recommended action, and

if the values of said interpretation reliability and said interpretation understanding for the current utterance in the second language are “

low” and

“

unclear,”

respectively, and the value of said interpretation reliability for the previous utterance in the first language is “

high,”

or if the values of said interpretation reliability and said interpretation understanding for the current utterance in the second language are “

high” and

“

unclear,”

respectively, and the values of said interpretation reliability and said interpretation understanding for the previous utterance in the first language are “

high” and

“

clear,”

respectively, presenting “

second speaker re-utters,”

“

second speaker checks recognition,”

“

first speaker looks up dictionary”

, and “

first speaker selects next translation candidate”

as the recommended actions.

27. The automatic interpreting program as set forth in claim 20, wherein

if the level of interpretation understanding for an utterance is “

unclear,”

then the current conversation status is determined, i.e., whether an error has occurred in the interpretation result of said utterance or a misunderstanding has occurred in the interpretation result of the previous utterance, by putting the combinations of said interpretation reliability and said interpretation understanding for said utterance and for the previous utterance in contrast with the pre-determined categories, and

actions to be presented as recommended actions are selected from options, including requesting re-utterance, re-uttering, requesting check of recognition, looking up dictionary, and selecting next translation candidate.

28. The automatic interpreting program as set forth in claim 20, wherein

if the level of interpretation understanding for an utterance is “

not understandable,”

then the current conversation status is determined, i.e., whether an error has occurred in the interpretation result of said utterance or a misunderstanding has occurred in the interpretation result of the previous utterance, by putting the combinations of said interpretation reliability and said interpretation understanding for said utterance and for the previous utterance in contrast with the pre-determined categories,

assuming that said interpretation reliability is categorized into two values, high and low, and said interpretation understanding is categorized into two values, clear and unclear, and if said interpretation understanding for an utterance is “

unclear,”

if the value of said interpretation reliability for said utterance is “

high” and

said interpretation reliability and said interpretation understanding for the previous utterance are “

low” and

“

clear,”

respectively, determining the conversation status to be a misunderstanding that has occurred in the interpretation result of the previous utterance and setting the conversation status,

if the value of said interpretation reliability for said utterance is “

high” and

the values of said interpretation reliability and said interpretation understanding for the previous utterance are “

high” and

“

clear,”

respectively, or if the value of said interpretation reliability for said utterance is “

low” and

the value of said interpretation understanding for the previous utterance is “

clear,”

determining the conversation status to be an error that has occurred in the interpretation result of said utterance and setting the conversation status, and

actions to be presented as recommended actions are selected from options, including requesting re-utterance, re-uttering, requesting check of recognition, looking up dictionary, and selecting next translation candidate.