System and method for translating chat messages

• US 10,769,387 B2
• Filed: 09/19/2018
• Issued: 09/08/2020
• Est. Priority Date: 09/21/2017
• Status: Active Grant

• Current Assignee: Mz Ip Holdings, LLC (AppLovin Corporation)
• Original Assignee: Mz Ip Holdings, LLC (AppLovin Corporation)
• Inventors: Wang, Pidong, Bojja, Nikhil, Guo, Shiman
• Primary Examiner(s): He, Jialong
• Application Number:
  US16/135,493
• Publication Number:
  US 20190087417A1
• Time in Patent Office: 720 Days
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models
• CPC Class Codes: G06F 40/53 Processing of non-Latin text kana-to-kanji conversion G06F40/129; vowelisation G06F40/232
• CPC Class Codes: G06F 40/55 Rule-based translation
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation

Abstract

Implementations of the present disclosure are directed to a method, a system, and an article for translating chat messages. An example method can include: receiving an electronic text message from a client device of a user; normalizing the electronic text message to generate a normalized text message; tagging at least one phrase in the normalized text message with a marker to generate a tagged text message, the marker indicating that the at least one phrase will be translated using a rule-based system; translating the tagged text message using the rule-based system and a machine translation system to generate an initial translation; and post-processing the initial translation to generate a final translation.

Claims

1. A method, comprising:

receiving an electronic text message from a client device of a user;

normalizing the electronic text message to generate a normalized text message comprising characters having a consistent width, the normalizing comprising at least one of;

converting a full-width character into a half-width character;

or

converting a half-width character into a full-width character;

tagging at least one phrase in the normalized text message with a marker to generate a tagged text message, the marker indicating that the at least one phrase will be translated using a rule-based system;

translating the tagged text message using the rule-based system and a machine translation system to generate an initial translation,

wherein translating the tagged text message comprises (i) translating the at least one phrase using the rule-based system and (ii) translating other words or phrases using the machine translation system, and

wherein the machine translation system is trained using training data comprising characters having a consistent width; and

post-processing the initial translation to generate a final translation.

2. The method of claim 1, wherein receiving the electronic text message comprises:

splitting the electronic text message into discrete sentences.

3. The method of claim 1, wherein normalizing the electronic text message comprises at least one of:

converting a control character into a space;

converting a UNICODE space character into an ASCII space character;

converting a Japanese character with Dakuten or Handakuten from a two-UNICODE point representation to a one-UNICODE point representation;

replacing an XML related character;

or

replacing a special character utilized by the machine translation system.

4. The method of claim 1, wherein the normalized message comprises characters having a consistent form.

5. The method of claim 1, wherein the marker comprises an XML, marker.

6. The method of claim 1, wherein tagging the at least one phrase comprises tokenizing the normalized text message into discrete words.

7. The method of claim 1, wherein tagging the at least one phrase comprises converting at least one upper case character in the normalized text message to a lower case character.

8. The method of claim 1, wherein the machine translation system comprises a statistical machine translator.

9. The method of claim 1, wherein post-processing the initial translation comprises at least one of:

detokenizing the initial translation;

removing the marker from the initial translation;

or

reintroducing into the initial translation a special character used by the machine translation system.

10. A system, comprising:

one or more computer processors programmed to perform operations comprising;

receiving an electronic text message from a client device of a user;

normalizing the electronic text message to generate a normalized text message comprising characters having a consistent width, the normalizing comprising at least one of;

converting a full-width character into a half-width character;

or

converting a half-width character into a full-width character;

tagging at least one phrase in the normalized text message with a marker to generate a tagged text message, the marker indicating that the at least one phrase will be translated using a rule-based system;

translating the tagged text message using the rule-based system and a machine translation system to generate an initial translation,

wherein translating the tagged text message comprises (i) translating the at least one phrase using the rule-based system and (ii) translating other words or phrases using the machine translation system, and

wherein the machine translation system is trained using training data comprising characters having a consistent width; and

post-processing the initial translation to generate a final translation.

11. The system of claim 10, wherein receiving the electronic text message comprises:

splitting the electronic text message into discrete sentences.

12. The system of claim 10, wherein normalizing the electronic text message comprises at least one of:

converting a control character into a space;

converting a UNICODE space character into an ASCII space character;

converting a Japanese character with Dakuten or Handakuten from a two-UNICODE point representation to a one-UNICODE point representation;

replacing an XML related character;

or

replacing a special character utilized by the machine translation system.

13. The system of claim 10, wherein the normalized message comprises characters having a consistent form.

14. The system of claim 10, wherein tagging the at least one phrase comprises tokenizing the normalized text message into discrete words.

15. The system of claim 10, wherein tagging the at least one phrase comprises converting at least one upper case character in the normalized text message to a lower case character.

16. The system of claim 10, wherein the machine translation system comprises a statistical machine translator.

17. The system of claim 10, wherein post-processing the initial translation comprises at least one of:

detokenizing the initial translation;

removing the marker from the initial translation;

or

reintroducing into the initial translation a special character used by the machine translation system.

18. An article, comprising:

a non-transitory computer-readable medium having instructions stored thereon that, when executed by one or more computer processors, cause the computer processors to perform operations comprising;

receiving an electronic text message from a client device of a user;

normalizing the electronic text message to generate a normalized text message comprising characters having a consistent width, the normalizing comprising at least one of;

converting a full-width character into a half-width character;

or

converting a half-width character into a full-width character;

tagging at least one phrase in the normalized text message with a marker to generate a tagged text message, the marker indicating that the at least one phrase will be translated using a rule-based system;

translating the tagged text message using the rule-based system and a machine translation system to generate an initial translation,

wherein translating the tagged text message comprises (i) translating the at least one phrase using the rule-based system and (ii) translating other words or phrases using the machine translation system, and

wherein the machine translation system is trained using training data comprising characters having a consistent width; and

post-processing the initial translation to generate a final translation.