LANGUAGE MODEL TRAINING METHOD AND DEVICE

• US 20170125013A1
• Filed: 08/19/2016
• Published: 05/04/2017
• Est. Priority Date: 10/29/2015
• Status: Abandoned Application

• Current Assignee: Le Holdings (Beijing) Co., Ltd., Le Shi Zhi Xin Electronic Technology (Tianjin) Limited
• Original Assignee: Le Holdings (Beijing) Co., Ltd., Le Shi Zhi Xin Electronic Technology (Tianjin) Limited
• Inventors: YAN, Zhiyong
• Application Number:
  US15/242,065
• Publication Number:
  US 20170125013A1
• Time in Patent Office: Days
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 2015/0633 using lexical or orthographic knowledge sources
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting

Abstract

The present disclosure provides a language model training method and device, including: obtaining a universal language model in an offline training mode, and clipping the universal language model to obtain a clipped language model; obtaining a log language model of logs within a preset time period in an online training mode; fusing the clipped language model with the log language model to obtain a first fusion language model used for carrying out first time decoding; and fusing the universal language model with the log language model to obtain a second fusion language model used for carrying out second time decoding. The method is used for solving the problem that a language model obtained offline in the prior art has poor coverage on new corpora, resulting in a reduced language recognition rate.

Claims

1. A language model training method, comprising:

obtaining a universal language model in an offline training mode, and clipping the universal language model to obtain a clipped language model;

obtaining a log language model of logs within a preset time period in an online training mode;

fusing the clipped language model with the log language model to obtain a first fusion language model used for carrying out first time decoding; and

fusing the universal language model with the log language model to obtain a second fusion language model used for carrying out second time decoding.

2. The method of claim 1, wherein the obtaining a log language model of logs within a preset time period in an online training mode comprises:

obtaining log information within the preset time period, filtering the log information, and carrying out word segmentation processing on the filtered log information to obtain a log model training corpus within the preset time period; and

training the log model training corpus to obtain the log language model.

3. The method of claim 1, wherein the clipped language model is a tri-gram language model, and correspondingly, the first fusion language model is a tri-gram fusion language model;

and

the universal language model is a tetra-gram language model, and correspondingly, the second fusion language model is a tetra-gram fusion language model.

4. The method of any one of claim 1, wherein the obtaining a universal language model in an offline training mode comprises:

collecting a model training corpus of each field;

for each field, training the model training corpus of the field to obtain the language model of the field; and

generating the collected language models corresponding to all fields into the universal language model in the interpolation mode.

5. The method of claim 4, wherein the clipping the universal language model to obtain a clipped language model comprises:

clipping the universal language model in a language model clipping mode based on entropy to obtain a second language model LM2;

clipping the second language model LM2 in the language model clipping mode based on entropy to obtain a third language model LM3; and

extracting the tri-gram language model from the third language model LM3, and clipping the extracted tri-gram language model to obtain the clipped language model LM4.

6. An electronic device, comprising:

at least one processor; and

a memory communicably connected with the at least one processor for storing instructions executable by the at least one processor, wherein execution of the instructions by the at least one processor causes the at least one processor to;

obtain a universal language model in an offline training mode;

clip the universal language model to obtain a clipped language model;

obtain a log language model of logs within a preset time period in an online training mode;

fuse the clipped language model with the log language model to obtain a first fusion language model used for carrying out first time decoding; and

fuse the universal language model with the log language model to obtain a second fusion language model used for carrying out second time decoding.

7. The device of claim 6, wherein the processor is further configured to perform the following steps:

obtaining log information within the preset time period, filtering the log information, and carrying out word segmentation processing on the filtered log information to obtain a log model training corpus within the preset time period; and

training the log model training corpus to obtain the log language model.

8. The device of claim 6, wherein the clipped language model is a tri-gram language model, and correspondingly, the first fusion language model is a tri-gram fusion language model;

and

the universal language model is a tetra-gram language model, and correspondingly, the second fusion language model is a tetra-gram fusion language model.

9. The device of claim 6, wherein the processor is further configured to perform the following steps:

collecting a model training corpus of each field;

for each field, training the model training corpus of the field to obtain the language model of the field; and

generating the collected language models corresponding to all fields into the universal language model in the interpolation mode.

10. The device of claim 9, wherein the processor is further configured to perform the following steps:

clipping the universal language model in a language model clipping mode based on entropy to obtain a second language model LM2;

clipping the second language model LM2 in the language model clipping mode based on entropy to obtain a third language model LM3; and

extracting the tri-gram language model from the third language model LM3, and clipping the extracted tri-gram language model to obtain the clipped language model LM4.

11. A non-transitory computer-readable storage medium storing executable instructions that, when executed by an electronic device with a touch-sensitive display, cause the electronic device to:

obtain a universal language model in an offline training mode;

clip the universal language model to obtain a clipped language model;

obtain a log language model of logs within a preset time period in an online training mode;

fuse the clipped language model with the log language model to obtain a first fusion language model used for carrying out first time decoding; and

fuse the universal language model with the log language model to obtain a second fusion language model used for carrying out second time decoding.

12. The non-transitory computer-readable storage medium of claim 11, wherein the electronic device is further configured to perform the following steps:

obtaining log information within the preset time period, filtering the log information, and carrying out word segmentation processing on the filtered log information to obtain a log model training corpus within the preset time period; and

training the log model training corpus to obtain the log language model.

13. The non-transitory computer-readable storage medium of claim 11, wherein the clipped language model is a tri-gram language model, and correspondingly, the first fusion language model is a tri-gram fusion language model;

and

the universal language model is a tetra-gram language model, and correspondingly, the second fusion language model is a tetra-gram fusion language model.

14. The non-transitory computer-readable storage medium of claim 11, wherein the electronic device is further configured to perform the following steps:

collecting a model training corpus of each field;

for each field, training the model training corpus of the field to obtain the language model of the field; and

generating the collected language models corresponding to all fields into the universal language model in the interpolation mode.

15. The non-transitory computer-readable storage medium of claim 14, wherein the electronic device is further configured to perform the following steps:

clipping the universal language model in a language model clipping mode based on entropy to obtain a second language model LM2;

clipping the second language model LM2 in the language model clipping mode based on entropy to obtain a third language model LM3; and

extracting the tri-gram language model from the third language model LM3, and clipping the extracted tri-gram language model to obtain the clipped language model LM4.