Technologies for end-of-sentence detection using syntactic coherence

• US 9,837,069 B2
• Filed: 12/22/2015
• Issued: 12/05/2017
• Est. Priority Date: 12/22/2015
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Shamir, Oren, Pereg, Oren, Wasserblat, Moshe, Mamou, Jonathan, Assayag, Michel
• Primary Examiner(s): Godbold, Douglas
• Application Number:
  US14/979,142
• Publication Number:
  US 20170178623A1
• Time in Patent Office: 714 Days
• Field of Search: None
• CPC Class Codes: G10L 15/04 Segmentation; Word boundary detection
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units
• CPC Class Codes: G10L 25/39 using genetic algorithms
• CPC Class Codes: G10L 25/87 Detection of discrete points within a voice signal

Abstract

Technologies for detecting an end of a sentence in automatic speech recognition are disclosed. An automatic speech recognition device may acquire speech data, and identify phonemes and words of the speech data. The automatic speech recognition device may perform a syntactic parse based on the recognized words, and determine an end of a sentence based on the syntactic parse. For example, if the syntactic parse indicates that a certain set of consecutive recognized words form a syntactically complete and correct sentence, the automatic speech recognition device may determine that there is an end of a sentence at the end of that set of words.

Claims

1. An automatic speech recognition device comprising:

a speech data capture module to acquire speech data;

a phoneme recognition module to recognize, based on the speech data, phonemes of the speech data;

a word recognition module to recognize, based on the phonemes, words of the speech data;

a syntactic parser module to parse, based on the words, the speech data to determine a syntactic coherence of the speech data; and

an end-of-sentence determination module to;

determine, based on the words, a word statistics end-of-sentence score;

determine, based on the syntactic coherence and the word statistics end-of-sentence score, an end of a sentence of the speech data; and

determine, based on the determined end of the sentence, a speech recognition result.

2. The automatic speech recognition device of claim 1, wherein to determine, based on the syntactic coherence, the end of the sentence comprises to determine, based on the syntactic coherence, the end of the sentence using a machine-learning-based algorithm.

3. The automatic speech recognition device of claim 1, wherein to determine, based on the syntactic coherence, the end of the sentence comprises to determine, based on the syntactic coherence, the end of the sentence using a rule-based algorithm.

4. The automatic speech recognition device of claim 1, wherein the end-of-sentence determination module is further to:

determine, based on the syntactic parse, a syntactic coherence end-of-sentence score, and

determine, based on the phonemes, an acoustic end-of-sentence score,

wherein to determine the end of the sentence comprises to determine the end of the sentence based on the syntactic coherence end-of-sentence score and the acoustic end-of-sentence score.

5. The automatic speech recognition device of claim 4, wherein to determine, based on the phonemes, the acoustic end-of-sentence score comprises to determine, based on a silence of the speech data, the acoustic end-of-sentence score.

6. The automatic speech recognition device of claim 4, wherein to determine the end of the sentence based on the syntactic coherence end-of-sentence score, the acoustic end-of-sentence score, and the word statistics end-of-sentence score comprises to determine the end of the sentence based on the syntactic coherence end-of-sentence score, the acoustic end-of-sentence score, and the word statistics end-of-sentence score using a machine-learning-based algorithm.

7. The automatic speech recognition device of claim 4, wherein to determine the end of the sentence based on the syntactic coherence end-of-sentence score, the acoustic end-of-sentence score, and the word statistics end-of-sentence score comprises to determine the end of the sentence based on the syntactic coherence end-of-sentence score, the acoustic end-of-sentence score, and the word statistics end-of-sentence score using a rule-based algorithm.

8. The automatic speech recognition device of claim 1, wherein the end-of-sentence determination module is further to:

determine, based on the syntactic parse, a syntactic coherence end-of-sentence score, and

determine, based on the acoustic features, an acoustic end-of-sentence score,

wherein to determine the end of the sentence comprises to determine the end of the sentence based on the syntactic coherence end-of-sentence score and the acoustic end-of-sentence score.

9. The automatic speech recognition device of claim 8, wherein the acoustic features comprise at least one of a frequency, a pitch, a rate of change of the pitch, an energy associated with one or more non-silent phonemes, or a rate of change of the energy.

10. One or more non-transitory, machine-readable storage media comprising a plurality of instructions stored thereon that, when executed, cause an automatic speech recognition device to:

acquire speech data;

recognize, based on the speech data, phonemes of the speech data;

recognize, based on the phonemes, words of the speech data;

parse, based on the words, the speech data to determine a syntactic coherence of the speech data;

determine, based on the words, a word statistics end-of-sentence score;

determine, based on the syntactic coherence and the word statistics end-of-sentence score, an end of a sentence of the speech data; and

determine, based on the determined end of the sentence, a speech recognition result.

11. The one or more non-transitory, machine-readable storage media of claim 10, wherein to determine, based on the syntactic coherence, the end of the sentence comprises to determine, based on the syntactic coherence, the end of the sentence using a machine-learning-based algorithm.

12. The one or more non-transitory, machine-readable storage media of claim 10, wherein to determine, based on the syntactic coherence, the end of the sentence comprises to determine, based on the syntactic coherence, the end of the sentence using a rule-based algorithm.

13. The one or more non-transitory, machine-readable storage media of claim 10, wherein the plurality of instructions further cause the automatic speech recognition device to:

determine, based on the syntactic parse, a syntactic coherence end-of-sentence score, and

determine, based on the phonemes, an acoustic end-of-sentence score,

wherein to determine the end of the sentence comprises to determine the end of the sentence based on the syntactic coherence end-of-sentence score and the acoustic end-of-sentence score.

14. The one or more non-transitory, machine-readable storage media of claim 13, wherein to determine, based on the phonemes, the acoustic end-of-sentence score comprises to determine, based on a silence of the speech data, the acoustic end-of-sentence score.

15. The one or more non-transitory, machine-readable storage media of claim 13, wherein to determine the end of the sentence based on the syntactic coherence end-of-sentence score, the acoustic end-of-sentence score, and the word statistics end-of-sentence score comprises to determine the end of the sentence based on the syntactic coherence end-of-sentence score, the acoustic end-of-sentence score, and the word statistics end-of-sentence score using a machine-learning-based algorithm.

16. The one or more non-transitory, machine-readable storage media of claim 10, wherein the plurality of instructions further cause the automatic speech recognition device to:

determine, based on the syntactic parse, a syntactic coherence end-of-sentence score, and

determine, based on the acoustic features, an acoustic end-of-sentence score,

wherein to determine the end of the sentence comprises to determine the end of the sentence based on the syntactic coherence end-of-sentence score and the acoustic end-of-sentence score.

17. The one or more non-transitory, machine-readable storage media of claim 16, wherein the acoustic features comprise at least one of a frequency, a pitch, a rate of change of the pitch, an energy associated with one or more non-silent phonemes, or a rate of change of the energy.

18. A method for determining an end of a sentence of speech data, the method comprising:

acquiring, by an automatic speech recognition device, speech data;

recognizing, by the automatic speech recognition device and based on the speech data, phonemes of the speech data;

recognizing, by the automatic speech recognition device and based on the phonemes, words of the speech data;

parsing, by the automatic speech recognition device and based on the words, the speech data to determine a syntactic coherence of the speech data;

determine, by the automatic speech recognition device and based on the words, a word statistics end-of-sentence score;

determining, by the automatic speech recognition device and based on the syntactic coherence and the word statistics end-of-sentence score, an end of a sentence of the speech data;

determining, by the automatic speech recognition device and based on the determined end of the sentence, a speech recognition result.

19. The method of claim 18, wherein determining the end of the sentence based on the syntactic coherence comprises determining the end of the sentence based on the syntactic coherence using a machine-learning-based algorithm.

20. The method of claim 18, wherein determining the end of the sentence based on the syntactic coherence comprises determining the end of the sentence based on the syntactic coherence using a rule-based algorithm.

21. The method of claim 18, further comprising:

determining, by the automatic speech recognition device, a syntactic coherence end-of-sentence score based on the syntactic parse, and

determining, by the automatic speech recognition device, an acoustic end-of-sentence score based on the phonemes,

wherein determining the end of the sentence comprises determining the end of the sentence based on the syntactic coherence end-of-sentence score and the acoustic end-of-sentence score.

22. The method of claim 21, wherein determining the acoustic end-of-sentence score based on the phonemes comprises determining the acoustic end-of-sentence score based on a silence of the speech data.

23. The method of claim 21, wherein determining the end of the sentence based on the syntactic coherence end-of-sentence score, the acoustic end-of-sentence score, and the word statistics end-of-sentence score comprises determining the end of the sentence based on the syntactic coherence end-of-sentence score, the acoustic end-of-sentence score, and the word statistics end-of-sentence score using a machine-learning-based algorithm.

24. The method of claim 18, further comprising:

determining, based on the syntactic parse, a syntactic coherence end-of-sentence score, and

determining, based on the acoustic features, an acoustic end-of-sentence score,

wherein determining the end of the sentence comprises determining the end of the sentence based on the syntactic coherence end-of-sentence score and the acoustic end-of-sentence score.

25. The method of claim 24, wherein the acoustic features comprise at least one of a frequency, a pitch, a rate of change of the pitch, an energy associated with one or more non-silent phonemes, or a rate of change of the energy.