SPEECH RECOGNITION SYSTEM AND METHOD USING INCREMENTAL DEVICE-BASED ACOUSTIC MODEL ADAPTATION

• US 20150081300A1
• Filed: 04/18/2014
• Published: 03/19/2015
• Est. Priority Date: 09/17/2013
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Kim, Dong-Hyun
• Granted Patent: US 9,601,112 B2
• Time in Patent Office: Days
• US Class Current: 704/246
• CPC Class Codes: G10L 15/07 to the speaker
• CPC Class Codes: G10L 2015/228 of application context

Abstract

An embodiment of the present invention relates to a speech recognition system and method using incremental device-based acoustic model adaptation. The speech recognition system comprises a model selection module selecting an acoustic model of multi-model tree by verifying and categorizing a device key transmitted from a user device; a model management module generating and incrementally adapting multi-model tree by categorizing voice data based on a user device; and a speech recognition module performing speech recognition by receiving the acoustic model selected from the model selection module and transmitting data of which reliability exceeds a predetermined threshold value to the model management module

Claims

1. A speech recognition system using incremental device-based acoustic model adaptation comprising:

a model selection module selecting an acoustic model of a multi-model tree by verifying and categorizing a device key transmitted from a user device;

a model management module generating and incrementally adapting the multi-model tree by categorizing voice data based on a user device; and

a speech recognition module performing speech recognition based on the acoustic model received from the model selection module and transmitting data of which reliability exceeds a predetermined threshold value to the model management module.

2. The speech recognition system of claim 1, wherein the model selection module comprises:

a device key verification part verifying a device key if it is a new device key or an existing device key and registering the device key to a device key tree which is a component of the multi-model tree when the device key is a new device key, or receiving an available acoustic model list of the device key tree from the multi-model tree when the device key is an existing device key; and

a model selecting part selecting one acoustic model or multi acoustic models to be used for parallel recognition from the acoustic model list and transmitting address and the device key information of the selected model to the speech recognition module.

3. The speech recognition system of claim 2, wherein the acoustic model list comprises sequentially a device independent model, a device dependent model, a detail device dependent model and a random ID dependent model included in the multi-model tree.

4. The speech recognition system of claim 2, wherein the model selecting part searches and selects acoustic models from a device node of an acoustic model tree which is a component of the multi-model tree using the device key, and selects acoustic models from an upper node connected to the device node if no acoustic model is present in the device node.

5. The speech recognition system of claim 4, wherein the model selecting part selects multi acoustic models from acoustic models of the user device corresponding to the device key and acoustic models of the upper node according to a predetermined rule in case of performing parallel recognition.

6. The speech recognition system of claim 1, wherein the model management module comprises:

a device-based multi-model tree generating part categorizing the voice data based on a user device, extracting features therefrom and forming the multi-model tree;

a multi-model tree categorized by a user device and comprising hierarchical acoustic models, incremental data and device key trees;

a data incrementing part receiving data of which the reliability exceeds a predetermined threshold value from the speech recognition module, incrementing phonetic information and data to the multi-model tree, evaluating degree of the incremented data of each node, and reporting the result to a model adaptation generating part when it exceeds a predetermined reference value; and

the model adaptation generating part incrementally adapting acoustic models corresponding to nodes where newly adapted data from the multi-model tree exceeds the predetermined reference value by using the information transmitted from the data incrementing part.

7. The speech recognition system of claim 6, wherein the device-based multi-model tree generating part categorizes the voice data by a user device and generates a multi-model tree which generates a device dependent model adapted at a device independent model.

8. The speech recognition system of claim 6, wherein, when a new device key is transferred from the model selection module, the multi-model tree registers the device key to the device key tree and generates a data node to collect data.

9. The speech recognition system of claim 6, wherein the data incrementing part searches the node registered in the multi-model tree by using the device key received from the speech recognition module and increments data to the searched node and its upper node.

10. The speech recognition system of claim 1, wherein the speech recognition module comprises:

a pre-processing part pre-processing speech signals inputted from the user device;

a recognition part performing speech recognition using the feature-extracted data provided from the pre-processing part and the acoustic model selected by the model selection module; and

a reliability determining part determining reliability of the speech recognition result received from the recognition part, outputting result with the highest reliability as a recognition result, and transmitting phonetic information, data and device key to the model management module when the reliability exceeds a threshold value.

11. The speech recognition system of claim 10, wherein the recognition part receives a plurality of acoustic models from the model selection module and performs parallel speech recognition for the plurality of acoustic models.

12. The speech recognition system of claim 1, wherein the speech recognition program is installed in a user device and the speech recognition program comprises:

a device key transferring part generating a device key with the initial installation of the speech recognition program and transmitting the device key to the model selection module, and transmitting the device key to the model selection module whenever the speech recognition program is used; and

a speech recording part recording voice data inputted when speech recognition is performed and transmitting the recorded voice data or the feature-extracted data to the speech recognition module.

13. The speech recognition system of claim 1, wherein the device key comprises a device name, a device detail name and a random ID.

14. The speech recognition system of claim 1, wherein the multi-model tree comprises:

a data tree in which the entire data is categorized based on device;

an acoustic model tree in which acoustic model of each node is generated by using the data in a node of the data tree as an adaption data; and

a device key tree in which a device key are registered.

15. A speech recognition method using incremental device-based acoustic model adaptation, the method comprising:

registering a device key transferred from a user device when a speech recognition program is installed to a device key tree;

selecting an available acoustic model by using the device key when the device key is transferred from the user device with execution of the speech recognition program;

performing speech recognition using the available acoustic model and the voice data transferred from the user device;

incrementing data, of which reliability exceeds a threshold value when the reliability of the speech recognition result is determined, with recognition information to a device node and an immediate upper node of multi-model tree according to the device key; and

performing acoustic model incremental adaptation when data of each node of the multi-model tree exceeds a predetermined reference value.