Device and method of simultaneous interpretation based on real-time extraction of interpretation unit

• US 10,366,173 B2
• Filed: 09/11/2017
• Issued: 07/30/2019
• Est. Priority Date: 09/09/2016
• Status: Active Grant

• Current Assignee: Hyundai Motor Company (Hyundai Motor Group), Kia Corp.
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Kim, Chang Hyun, Kim, Young Kil, Lee, Yun Keun
• Primary Examiner(s): Chawan, Vijay B
• Application Number:
  US15/700,537
• Publication Number:
  US 20180075023A1
• Time in Patent Office: 687 Days
• Field of Search: 704 2, 704 9, 704 10, 704270, 704275, 704259, 704232, 704235, 705 1449, 706 12, 707769
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 40/103 Formatting, i.e. changing of presentation of documents automatic justification G06F40/189; automatic line break hyphenation G06F40/191
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/10 using distance or distortion measures between unknown speech and reference templates
• CPC Class Codes: G10L 15/16 using artificial neural networks
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10

Abstract

The present invention relates to a device of simultaneous interpretation based on real-time extraction of an interpretation unit, the device including a voice recognition module configured to recognize voice units as sentence units or translation units from vocalized speech that is input in real time, a real-time interpretation unit extraction module configured to form one or more of the voice units into an interpretation unit, and a real-time interpretation module configured to perform an interpretation task for each interpretation unit formed by the real-time interpretation unit extraction module.

Claims

1. A device of simultaneous interpretation based on real-time extraction of an interpretation unit, the device comprising:

a voice recognition module configured to recognize voice units as sentence units or translation units from vocalized speech that is input in real time;

a real-time interpretation unit extraction module configured to form one or more of the voice units into an interpretation unit; and

a real-time interpretation module configured to perform an interpretation task for each interpretation unit formed by the real-time interpretation unit extraction module.

2. The device of claim 1, wherein the real-time interpretation unit extraction module includes:

an input buffer manager configured to store a voice unit that is input and a remaining voice unit that is not yet included in sentences to be translated due to a previous interpretation unit extraction result;

a unit-separating morpheme analyzer configured to detect morphemes of each of the voice units;

a voice unit separator configured to re-separate the voice units according to a morpheme analysis result of the unit-separating morpheme analyzer; and

an interpretation unit former configured to form an interpretation unit by combining a current voice unit and a previous voice unit.

3. The device of claim 2, wherein the voice unit separator re-separates a voice unit stored in the input buffer manager on the basis of a lexical characteristic, a morphological characteristic, an acoustic characteristic, and a time characteristic.

4. The device of claim 3, wherein the lexical characteristic is a characteristic of determining whether a word that is able to be determined as a beginning of a sentence in a language is present.

5. The device of claim 3, wherein the morphological characteristic is a characteristic of determining whether a predicate or a final ending is included after acquiring a morpheme analysis result related to the voice unit.

6. The device of claim 3, wherein the acoustic characteristic is a characteristic related to pause information and prosody and stress information.

7. The device of claim 1, wherein the real-time interpretation module performs translation by using both a module-based method based on modules for morpheme analysis, structure analysis, conversion, and transition word generation and a mechanical learning method using statistical machine translation (SMT), a deep neural network (DNN), and the like.

8. The device of claim 7, further comprising a context management module configured to store all previous interpretation units and results of analyzing and generating morphemes/structures and translation results related to the interpretation units.

9. A method of simultaneous interpretation based on real-time extraction of an interpretation unit, the method comprising:

recognizing, by a voice recognition module, voice units as sentence units or translation units from vocalized speech that is input in real time;

forming, by a real-time interpretation unit extraction module, one or more of the voice units into an interpretation unit; and

performing, by a real-time interpretation module, an interpretation task for each interpretation unit formed by the real-time interpretation unit extraction module.

10. The method of claim 9, wherein the forming of the interpretation unit with the voice units includes:

storing, by an input buffer manager, a voice unit that is input and a remaining voice unit that is not yet included in sentences to be translated due to a previous interpretation unit extraction result;

detecting, by a unit-separating morpheme analyzer, morphemes of each of the voice units;

re-separating, by a voice unit separator, the voice units according to a morpheme analysis result of the unit-separating morpheme analyzer; and

forming, by an interpretation unit former, an interpretation unit by combining a current voice unit and a previous voice unit.

11. The method of claim 10, wherein the re-separating of the voice units includes re-separating a voice unit stored in the input buffer manager on the basis of a lexical characteristic, a morphological characteristic, an acoustic characteristic, and a time characteristic.

12. The method of claim 11, wherein the lexical characteristic is a characteristic of determining whether a word that is able to be determined as a beginning of a sentence in a language is present.

13. The method of claim 11, wherein the morphological characteristic is a characteristic of determining whether a predicate or a final ending is included after acquiring a morpheme analysis result related to the voice unit.

14. The method of claim 11, wherein the acoustic characteristic is a characteristic related to pause information and prosody and stress information.

15. The method of claim 9, wherein the performing of the interpretation task for each interpretation unit includes performing translation by using both a module-based method based on modules for morpheme analysis, structure analysis, conversion, and transition word generation and a mechanical learning method using SMT, a DNN, and the like.

16. The method of claim 15, wherein the performing of the interpretation task for each interpretation unit includes storing all previous interpretation units and results of analyzing and generating morphemes/structures and translation results related to the interpretation units.