Method of translating speech signal and electronic device employing the same

• US 10,614,170 B2
• Filed: 09/25/2017
• Issued: 04/07/2020
• Est. Priority Date: 09/26/2016
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Original Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Inventors: Kim, Sang-ha, Kim, Eun-kyoung, Yu, Ji-sang, Ryu, Jong-youb, Park, Chi-youn, Lee, Jin-sik, Lee, Jae-won
• Primary Examiner(s): Sharma, Neeraj
• Application Number:
  US15/714,249
• Publication Number:
  US 20180089176A1
• Time in Patent Office: 925 Days
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A method of translating a first language-based speech signal into a second language is provided. The method includes receiving the first language-based speech signal, converting the first language-based speech signal into a first language-based text including non-verbal information, by performing voice recognition on the first language-based speech signal, and translating the first language-based text into the second language, based on the non-verbal information.

Claims

1. A method of translating a first language-based speech signal into a second language, the method comprising:

receiving the first language-based speech signal;

converting the first language-based speech signal into a first language-based text including non-verbal information, by performing voice recognition on the first language-based speech signal; and

translating the first language-based text into the second language, based on the non-verbal information,

wherein the converting of the first language-based speech signal into the first language-based text including the non-verbal information by performing voice recognition on the first language-based speech signal comprises;

acquiring at least one first language-based candidate sequence corresponding to the speech signal;

excluding non-verbal information included in the acquired at least one first language-based candidate sequence from each of the acquired at least one first language-based candidate sequence;

acquiring probability information based on a language model, with respect to each of the at least one first language-based candidate sequence excluding the non-verbal information;

determining at least one from among the at least one first language-based candidate sequence to be the first language-based text, based on the acquired probability information; and

adding the excluded non-verbal information to the first language-based text, based on information about a condition for detecting the non-verbal information from the speech signal, and

wherein the first language-based text including the non-verbal information is translated into the second language.

2. The method of claim 1, wherein the non-verbal information comprises at least one of information about emphasized vocabularies, information about sentence patterns, information about non-standard words, information about pauses, or information about hesitation words.

3. The method of claim 1, wherein the translating of the first language-based text into the second language comprises:

acquiring at least one section divided from the first language-based text;

determining probability information of the at least one section, based on the non-verbal information; and

translating the first language-based text into the second language, based on the probability information.

4. The method of claim 3, wherein the determining of the probability information comprises:

combining at least one section and a section adjacent to the at least one section; and

based on whether the combined section includes the non-verbal information, determining the probability information of the combined section.

5. The method of claim 3, wherein the determining of the probability information comprises:

detecting a section including a hesitation word from the at least one section, based on the non-verbal information; and

determining probability information of a section including the hesitation word such that the hesitation word is translated into the hesitation word of the second language.

6. The method of claim 3, wherein the determining of the probability information comprises:

excluding a section including a hesitation word from the at least one section, based on the non-verbal information; and

determining probability information of the at least one section from which the section including the hesitation word is excluded.

7. An electronic device for translating a first language-based speech signal into a second language, the electronic device comprising:

a receiver configured to receive the first language-based speech signal;

a processor coupled to the receiver and an outputter, and configured to;

convert the first language-based speech signal into a first language-based text including non-verbal information, by performing voice recognition on the first language-based speech signal, and

translate the first language-based text into the second language, based on the non-verbal information; and

the outputter configured to output a result of the translation into the second language,

wherein, to convert the first language-based speech signal into a first language-based text including non-verbal information by performing voice recognition on the first language-based speech signal, the processor is further configured to;

acquire at least one first language-based candidate sequence corresponding to the speech signal,

exclude non-verbal information included in the acquired at least one first language-based candidate sequence from each of the acquired at least one first language-based candidate sequence,

acquire probability information based on a language model, with respect to each of the at least one first language-based candidate sequence excluding the non-verbal information,

determine at least one from among the at least one first language-based candidate sequence to be the first language-based text, based on the acquired probability information, and

add the excluded non-verbal information to the first language-based text, based on information about a condition for detecting the non-verbal information from the speech signal, and

wherein the first language-based text including the non-verbal information is translated into the second language.

8. The electronic device of claim 7, wherein the non-verbal information comprises at least one of information about emphasized vocabularies, information about sentence patterns, information about non-standard words, information about pauses, or information about hesitation words.

9. The electronic device of claim 7, wherein the processor is further configured to:

acquire at least one section divided from the first language-based text,

determine probability information of the at least one section based on the non-verbal information, and

translate the first language-based text into the second language, based on the probability information.

10. The electronic device of claim 9, wherein the processor is further configured to:

combine at least one section and a section adjacent to the at least one section, and

based on whether the combined section includes the non-verbal information, determine probability information of the combined section.

11. The electronic device of claim 9, wherein the processor is further configured to:

identify a section including a hesitation word from the at least one section, based on the non-verbal information, and

determine probability information of a section including the hesitation word such that the hesitation word is translated into the hesitation word of the second language.

12. The electronic device of claim 9, wherein the processor is further configured to:

exclude a section including a hesitation word from the at least one section, based on the non-verbal information, and

determine probability information of the at least one section from which the section including the hesitation word is excluded.

13. The electronic device of claim 7, wherein the receiver comprises at least one of a microphone, a communication circuit, or a camera.

14. The electronic device of claim 7, wherein the outputter comprises at least one of a speaker, a communication circuit, or a display.

15. A non-transitory computer-readable recording medium having recorded thereon a program, which when executed by a processor, causes the processor to:

control to receive a first language-based speech signal;

convert the first language-based speech signal into a first language-based text including non-verbal information, by performing voice recognition on the first language-based speech signal; and

translate the first language-based text into a second language, based on the non-verbal information,

wherein, to convert the first language-based speech signal into a first language-based text including non-verbal information by performing voice recognition on the first language-based speech signal, the program further causes the processor to;

acquire at least one first language-based candidate sequence corresponding to the speech signal;

exclude non-verbal information included in the acquired at least one first language-based candidate sequence from each of the acquired at least one first language-based candidate sequence;

acquire probability information based on a language model, with respect to each of the at least one first language-based candidate sequence excluding the non-verbal information;

determine at least one from among the at least one first language-based candidate sequence to be the first language-based text, based on the acquired probability information; and

add the excluded non-verbal information to the first language-based text, based on information about a condition for detecting the non-verbal information from the speech signal, and

wherein the first language-based text including the non-verbal information is translated into the second language.

16. The non-transitory computer-readable recording medium of claim 15, wherein the program, which when executed by the processor, further causes the processor to:

acquire at least one section divided from the first language-based text;

determine probability information of the at least one section based on the non-verbal information; and

translate the first language-based text into the second language, based on the probability information.