APPARATUS AND METHOD OF ACOUSTIC SCORE CALCULATION AND SPEECH RECOGNITION

• US 20170025119A1
• Filed: 06/20/2016
• Published: 01/26/2017
• Est. Priority Date: 07/24/2015
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: SONG, Inchul, CHOI, Young Sang
• Granted Patent: US 10,714,077 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/14 using statistical models, e.g. Hidden Markov Models [HMMs] G10L15/18 takes precedence
• CPC Class Codes: G10L 15/16 using artificial neural networks
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams

Abstract

An apparatus for calculating acoustic score, a method of calculating acoustic score, an apparatus for speech recognition, a method of speech recognition, and an electronic device including the same are provided. An apparatus for calculating acoustic score includes a preprocessor configured to sequentially extract audio frames into windows and a score calculator configured to calculate an acoustic score of a window by using a deep neural network (DNN)-based acoustic model.

Claims

1. An apparatus for calculating acoustic score, the apparatus comprising:

a preprocessor configured to sequentially extract audio frames into windows; and

a score calculator configured to calculate an acoustic score of a window by using a deep neural network (DNN)-based acoustic model.

2. The apparatus of claim 1, wherein the DNN is a bidirectional recurrent deep neural network (BRDNN).

3. The apparatus of claim 1, further comprising a padding adder configured to add padding frames to both sides of the window.

4. The apparatus of claim 3, wherein the padding frames added to the window overlap with one or more frames of a window adjacent to the window.

5. The apparatus of claim 4, wherein, in response to an acoustic score of the window being calculated, the apparatus re-calculates an acoustic score of the overlapping frames of the window by using a statistical method based on a pre-calculated acoustic score of the padding frames of the adjacent window.

6. The apparatus of claim 5, wherein the statistical method comprises calculating at least one of an arithmetic mean and a geometric mean.

7. The apparatus of claim 3, wherein a size of the window or a size of the padding frames is determined based on one or more of a speech recognition application field and computing performance capability of a device applied with the apparatus

8. A method of calculating an acoustic score, the method comprising:

sequentially extracting audio frames into windows; and

calculating an acoustic score of a window by using a deep neural network (DNN)-based acoustic model.

9. The method of claim 8, wherein the DNN is a bidirectional recurrent deep neural network (BRDNN).

10. The method of claim 8, further comprising adding padding frames to both sides of the window.

11. The method of claim 10, wherein the padding frames added to the window overlap with one or more frames of a window adjacent to the window.

12. The method of claim 11, further comprising re-calculating an acoustic score of the overlapping frames of the window by using a statistical method based on a pre-calculated acoustic score of the padding frames of the adjacent window.

13. The method of claim 12, wherein the statistical method comprises at least one of an arithmetic mean and a geometric mean.

14. The method of claim 11, wherein a size of the window or a size of the padding frames is determined based on one or more of a speech recognition application field and computing performance capability of a device performing the method.

15. A speech recognition apparatus, comprising:

a preprocessor configured to extract frames of an audio signal into windows while the frames are successively input;

an acoustic model applier configured to output an acoustic score of a window by using a deep neural network (DNN)-based acoustic model; and

a decoder configured to output an incremental recognition result, obtained up to a current window, based on the output acoustic score of the window.

16. The apparatus of claim 15, further comprising a language model applier configured to output a word/sentence score by using a language model, wherein the decoder outputs the incremental recognition result, obtained up to the current window, further based on the output word/sentence score.

17. The apparatus of claim 15, wherein in response to predetermined criteria being satisfied, the decoder outputs the incremental recognition result obtained up to the current window as a final recognition result of the audio signal.

18. The apparatus of claim 15, wherein the DNN is a bidirectional recurrent deep neural network (BRDNN).

19. The apparatus of claim 16, wherein the language model is based on an n-gram or a neural network.

20. The apparatus of claim 15, wherein the acoustic model applier adds padding frames to both sides of the window, and calculates the acoustic score of each frame of the window, to which the padding frames are added, wherein the acoustic score of the window is calculated by considering a pre-calculated acoustic score of the padding frames of a window adjacent to the window.

21. A speech recognition method, comprising:

extracting frames of an audio signal to be recognized in a window while the frames are successively input;

outputting an acoustic score of the window by using a deep neural network (DNN)-based acoustic model; and

outputting an incremental recognition result, obtained up to a current window, based on the output acoustic score of the window.

22. The method of claim 21, further comprising outputting a word/sentence score by using a language model, wherein the outputting of the incremental recognition result comprises outputting the incremental recognition result, obtained up to the current window, further based on the output word/sentence score.

23. The method of claim 21, further comprising:

determining whether predetermined criteria are satisfied; and

in response to a determination that the predetermined criteria are satisfied, outputting the incremental recognition result obtained up to the current window as a final recognition result of the audio signal.

24. The method of claim 21, further comprising adding padding frames on both sides of the window, wherein the outputting of the acoustic score comprises calculating the acoustic score of each frame of the window, to which the padding frames are added, by considering a pre-calculated acoustic score of the padding frames of a window adjacent to the window.

25. An electronic device, comprising:

a speech input device configured to receive an audio signal from a user;

a speech recognizer configured to recognize the audio signal of the user by outputting an acoustic score of frames of the input audio signal in a window by using a deep neural network (DNN)-based acoustic model while the audio signal is input, and by incrementally decoding the output acoustic score of the window; and

a processor configured to perform a predetermined operation based on the recognition result.

26. The electronic device of claim 25, wherein the operation comprises at least one of outputting the recognition result in voice or in a text format, translation of the recognition result into another language, and processing of commands for controlling the electronic device.