Linear scoring for low power wake on voice

• US 10,083,689 B2
• Filed: 12/23/2016
• Issued: 09/25/2018
• Est. Priority Date: 12/23/2016
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Bocklet, Tobias, Dorau, Tomasz, Sobon, Przemyslaw, Tomaszewski, Przemyslaw
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): Le, Thuykhanh
• Application Number:
  US15/390,384
• Publication Number:
  US 20180182388A1
• Time in Patent Office: 641 Days
• Field of Search: None
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/16 using artificial neural networks
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Key phrase detection techniques for applications such as wake on voice are discussed include performing a vectorized operation on a multiple element acoustic score vector for a current time instance including a single state rejection model score and scores for a multiple state key phrase model and a multiple element state score vector for a previous time instance including a previous state score for the single state rejection model and previous state scores for the multiple state key phrase model to generate a multiple element score summation vector and a second vectorized operation on the multiple element score summation vector to determine a multiple element state score vector for the current time instance. The multiple element state score vector for the current time instance may then be evaluated to determine whether received audio input includes a key phrase corresponding to the multiple state key phrase model.

Claims

1. A computer-implemented method for key phrase detection comprising:

generating a multiple element acoustic score vector for a current time instance based on received audio input, wherein the multiple element acoustic score vector comprises at least an acoustic score for at least one single state rejection model and acoustic scores for at least one multiple state key phrase model, and wherein the multiple state key phrase model corresponds to a predetermined key phrase;

receiving a multiple element state score vector for a previous time instance, wherein the multiple element state score vector comprises a previous state score for the single state rejection model and previous state scores for the multiple state key phrase model;

performing a vectorized operation to add the multiple element acoustic score vector and the multiple element state score vector to generate a multiple element score summation vector;

determining a second multiple element state score vector for the current time instance based on the multiple element score summation vector, wherein the second multiple element state score vector comprises a current state score for the single state rejection model and current state scores for the multiple state key phrase model;

evaluating the current state score for the single state rejection model and a final state score for the multiple state key phrase model to detect the predetermined key phrase in the received audio input; and

providing at least one of a system wake indicator or a system command in response to the detected predetermined key phrase.

2. The method of claim 1, wherein the multiple element score summation vector comprises a rejection state value corresponding to a sum of the acoustic score for the single state rejection model and the previous state score for the single state rejection model followed by subsequent key phrase model values corresponding to sums of acoustic scores for the multiple state key phrase model and previous state scores for the multiple state key phrase model, and wherein determining the second multiple element state score vector for the current time instance based on the multiple element score summation vector comprises:

performing a second vectorized operation to determine a maximum of the rejection state value and a first value of the key phrase model values and at least a maximum of the first value and a second value of the key phrase model values.

3. The method of claim 2, wherein the first value corresponds to an initial state of the multiple state key phrase model.

4. The method of claim 2, wherein performing the second vectorized operation further determines maxima between adjacent remaining values of the key phrase model values to provide the current state scores for the multiple state key phrase model.

5. The method of claim 2, wherein the vectorized operation adds corresponding elements of the multiple element acoustic score vector and the multiple element state score vector simultaneously and the second vectorized operation determines the maximum of the rejection state value and the first value of the key phrase model values and at least the maximum of the first value and the second value of the key phrase model values simultaneously.

6. The method of claim 1, wherein generating the multiple element acoustic score vector for the current time instance comprises:

determining the score for the single state rejection model as a maximum of a best rejection score corresponding to the single state rejection model and a best silence score corresponding to the single state rejection model; and

accessing a deep neural network acoustic model to determine the scores for the multiple state key phrase model.

7. The method of claim 1, wherein generating the multiple element acoustic score vector for the current time instance comprises:

updating, for a silence state of the key phrase model, a first score of the scores for the multiple state key phrase model corresponding to the silence state with a best silence score when the best silence score is greater than a current acoustic score of the silence state.

8. The method of claim 1, further comprising:

determining, for a rejection model state of the key phrase model, a first rejection score as a sum of an acoustic score from the multiple element acoustic score vector corresponding to the rejection model transition state and an element state score from the multiple element state score vector corresponding to the rejection model transition state; and

updating, prior to evaluating the current state score for the single state rejection model and the final state score for the multiple state key phrase model to determine whether the received audio input is associated with the predetermined key phrase, the current state score for the single state rejection model with the maximum of the first rejection score and the previously determined current state score.

9. The method of claim 1, wherein the second multiple element state score vector further comprises second current state scores for a second multiple state key phrase model corresponding to a second predetermined key phrase and a spare state between the multiple state key phrase model and the second multiple state key phrase model, the method further comprising:

determining, prior to evaluating the current state score for the single state rejection model and the final state score for the multiple state key phrase model to determine whether the received audio input is associated with the predetermined key phrase, a maximum of the final state score for the multiple state key phrase model and a second final state score for the second multiple state key phrase model,

wherein evaluating the current state score for the single state rejection model and the final state score for the multiple state key phrase model to determine whether the received audio input is associated with the predetermined key phrase is performed when the final state score is the maximum.

10. The method of claim 1, wherein evaluating the current state score for the single state rejection model and the final state score for the multiple state key phrase model to determine whether the received audio input is associated with the predetermined key phrase comprises:

determining a log likelihood score based on the current state score for the single state rejection model and the final state score for the multiple state key phrase model; and

comparing the log likelihood score to a threshold.

11. A system for performing key phrase detection comprising:

a memory configured to store a multiple element state score vector for a previous time instance, wherein the multiple element state score vector comprises a previous state score for at least one single state rejection model and previous state scores for at least one multiple state key phrase model, wherein the multiple state key phrase model corresponds to a predetermined key phrase; and

a digital signal processor coupled to the memory, the digital signal processor to generate a multiple element acoustic score vector for a current time instance based on received audio input, wherein the multiple element acoustic score vector comprises at least an acoustic score for the single state rejection model and scores for the multiple state key phrase model, to receive the multiple element state score vector for the previous time instance from the memory, to perform a vectorized operation to add the multiple element acoustic score vector and the multiple element state score vector to generate a multiple element score summation vector, to determine a second multiple element state score vector for the current time instance based on the multiple element score summation vector, wherein the second multiple element state score vector comprises a current state score for the single state rejection model and current state scores for the multiple state key phrase model, to evaluate the current state score for the single state rejection model and a final state score for the multiple state key phrase model to detect the predetermined key phrase in the received audio input, and to provide at least one of a system wake indicator or a system command in response to the detected predetermined key phrase.

12. The system of claim 11, wherein the multiple element score summation vector comprises a rejection state value corresponding to a sum of the acoustic score for the single state rejection model and the previous state score for the single state rejection model followed by subsequent key phrase model values corresponding to sums of acoustic scores for the multiple state key phrase model and previous state scores for the multiple state key phrase model, and wherein to determine the second multiple element state score vector for the current time instance based on the multiple element score summation vector comprises the digital signal processor to perform a second vectorized operation to determine a maximum of the rejection state value and a first value of the key phrase model values and at least a maximum of the first value and a second value of the key phrase model values.

13. The system of claim 12, wherein to perform the second vectorized operation further comprises the digital signal processor to determine maxima between adjacent remaining values of the key phrase model values to provide the current state scores for the multiple state key phrase model.

14. The system of claim 12, wherein the vectorized operation adds corresponding elements of the multiple element acoustic score vector and the multiple element state score vector simultaneously and the second vectorized operation determines the maximum of the rejection state value and the first value of the key phrase model values and at least the maximum of the first value and the second value of the key phrase model values simultaneously.

15. The system of claim 11, wherein to generate the multiple element acoustic score vector for the current time instance comprises the digital signal processor to update, for a silence state of the key phrase model, a first score of the scores for the multiple state key phrase model corresponding to the silence state with a best silence score when the best silence score is greater than a current acoustic score of the silence state.

16. The system of claim 11, the digital signal processor is further to determine, for a rejection model state of the key phrase model, a first rejection score as a sum of an acoustic score from the multiple element acoustic score vector corresponding to the rejection model transition state and an element state score from the multiple element state score vector corresponding to the rejection model transition state and to update, prior to evaluating the current state score for the single state rejection model and the final state score for the multiple state key phrase model to determine whether the received audio input is associated with the predetermined key phrase, the current state score for the single state rejection model with the maximum of the first rejection score and the previously determined current state score.

17. The system of claim 11, wherein the second multiple element state score vector further comprises second current state scores for a second multiple state key phrase model corresponding to a second predetermined key phrase and a spare state between the multiple state key phrase model and the second multiple state key phrase model, the digital signal processor further to determine, prior to evaluating the current state score for the single state rejection model and the final state score for the multiple state key phrase model to determine whether the received audio input is associated with the predetermined key phrase, a maximum of the final state score for the multiple state key phrase model and a second final state score for the second multiple state key phrase model, wherein evaluating the current state score for the single state rejection model and the final state score for the multiple state key phrase model to determine whether the received audio input is associated with the predetermined key phrase is performed when the final state score is the maximum.

18. The system of claim 11, wherein the memory and the digital signal processor are implemented as a system on a chip.

19. At least one non-transitory computer readable medium comprising a plurality of instruction that, in response to being executed on a device, cause the device to perform key phrase detection by:

generating a multiple element acoustic score vector for a current time instance based on received audio input, wherein the multiple element acoustic score vector comprises at least an acoustic score for at least one single state rejection model and acoustic scores for at least one multiple state key phrase model, and wherein the multiple state key phrase model corresponds to a predetermined key phrase;

receiving a multiple element state score vector for a previous time instance, wherein the multiple element state score vector comprises a previous state score for the single state rejection model and previous state scores for the multiple state key phrase model;

performing a vectorized operation to add the multiple element acoustic score vector and the multiple element state score vector to generate a multiple element score summation vector;

determining a second multiple element state score vector for the current time instance based on the multiple element score summation vector, wherein the second multiple element state score vector comprises a current state score for the single state rejection model and current state scores for the multiple state key phrase model;

evaluating the current state score for the single state rejection model and a final state score for the multiple state key phrase model to detect the predetermined key phrase in the received audio input; and

providing at least one of a system wake indicator or a system command in response to the detected predetermined key phrase.

20. The non-transitory computer readable medium of claim 19, wherein the multiple element score summation vector comprises a rejection state value corresponding to a sum of the acoustic score for the single state rejection model and the previous state score for the single state rejection model followed by subsequent key phrase model values corresponding to sums of acoustic scores for the multiple state key phrase model and previous state scores for the multiple state key phrase model, and wherein determining the second multiple element state score vector for the current time instance based on the multiple element score summation vector comprises:

performing a second vectorized operation to determine a maximum of the rejection state value and a first value of the key phrase model values and at least a maximum of the first value and a second value of the key phrase model values.

21. The non-transitory computer readable medium of claim 20, wherein performing the second vectorized operation further determines maxima between adjacent remaining values of the key phrase model values to provide the current state scores for the multiple state key phrase model.

22. The non-transitory computer readable medium of claim 20, wherein performing the second vectorized corresponding elements of the multiple element acoustic score vector and the multiple element state score vector simultaneously and the second vectorized operation determines the maximum of the rejection state value and the first value of the key phrase model values and at least the maximum of the first value and the second value of the key phrase model values simultaneously.

23. The non-transitory computer readable medium of claim 19, wherein generating the multiple element acoustic score vector for the current time instance comprises:

updating, for a silence state of the key phrase model, a first score of the scores for the multiple state key phrase model corresponding to the silence state with a best silence score when the best silence score is greater than a current acoustic score of the silence state.

24. The non-transitory computer readable medium of claim 19, further comprising instruction that, in response to being executed on the device, cause the device to perform key phrase detection by:

determining, for a rejection model state of the key phrase model, a first rejection score as a sum of an acoustic score from the multiple element acoustic score vector corresponding to the rejection model transition state and an element state score from the multiple element state score vector corresponding to the rejection model transition state; and

updating, prior to evaluating the current state score for the single state rejection model and the final state score for the multiple state key phrase model to determine whether the received audio input is associated with the predetermined key phrase, the current state score for the single state rejection model with the maximum of the first rejection score and the previously determined current state score.

25. The non-transitory computer readable medium of claim 19, wherein the second multiple element state score vector further comprises second current state scores for a second multiple state key phrase model corresponding to a second predetermined key phrase and a spare state between the multiple state key phrase model and the second multiple state key phrase model, the machine readable medium further comprising instructions that, in response to being executed on the device, cause the device to perform key phrase detection by:

determining, prior to evaluating the current state score for the single state rejection model and the final state score for the multiple state key phrase model to determine whether the received audio input is associated with the predetermined key phrase, a maximum of the final state score for the multiple state key phrase model and a second final state score for the second multiple state key phrase model,

wherein evaluating the current state score for the single state rejection model and the final state score for the multiple state key phrase model to determine whether the received audio input is associated with the predetermined key phrase is performed when the final state score is the maximum.