Keyword model generation for detecting user-defined keyword

• US 9,953,632 B2
• Filed: 08/22/2014
• Issued: 04/24/2018
• Est. Priority Date: 04/17/2014
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Yun, Sungrack, Kim, Taesu
• Primary Examiner(s): Siddo, Ibrahim
• Application Number:
  US14/466,644
• Publication Number:
  US 20150302847A1
• Time in Patent Office: 1,341 Days
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/07 to the speaker
• CPC Class Codes: G10L 2015/027 Syllables being the recognition units
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/226 using non-speech characteristics

Abstract

According to an aspect of the present disclosure, a method for generating a keyword model of a user-defined keyword in an electronic device is disclosed. The method includes receiving at least one input indicative of the user-defined keyword, determining a sequence of subwords from the at least one input, generating the keyword model associated with the user-defined keyword based on the sequence of subwords and a subword model of the subwords, wherein the subword model is configured to model a plurality of acoustic features of the subwords based on a speech database, and providing the keyword model associated with the user-defined keyword to a voice activation unit configured with a keyword model associated with a predetermined keyword.

Claims

1. A method, performed in an electronic device, for generating a keyword model associated with a user-defined keyword, the method comprising:

receiving a speech input from a user, at a sensor of the electronic device, in response to a prompt;

extracting a speech segment from the speech input;

determining a sequence of subwords representative of the speech segment of the speech input;

obtaining at least one subword model based on the sequence of subwords, wherein the at least one subword model includes a plurality of subword units and model parameters associated with each of the plurality of subword units;

generating the keyword model associated with the user-defined keyword by;

modifying the plurality of subword units of the at least one subword model based on the sequence of subwords representative of the speech segment of the speech input; and

adjusting the model parameters of the at least one subword model based on acoustic features extracted from the speech segment of the speech input; and

providing the keyword model associated with the user-defined keyword to a voice activation unit.

2. The method of claim 1, further comprising:

receiving an input sound; and

detecting, based on the keyword model associated with the user-defined keyword, the user-defined keyword in the input sound.

3. The method of claim 2, further comprising performing a function associated with the user-defined keyword or a predetermined keyword.

4. The method of claim 1, wherein adjusting the model parameters based on the speech input is performed prior to generating the keyword model associated with the user-defined keyword.

5. The method of claim 1, wherein generating the keyword model associated with the user-defined keyword comprises determining a threshold score of the keyword model associated with the user-defined keyword.

6. The method of claim 5, wherein the threshold score for the keyword model associated with the user-defined keyword is adjusted by:

receiving a test input sound indicative of the user-defined keyword;

determining a matching score of the test input sound based on the keyword model associated with the user-defined keyword; and

adjusting the threshold score of the keyword model associated with the user-defined keyword based on the matching score.

7. The method of claim 1, further comprising storing the keyword model associated with the user-defined keyword at a storage unit.

8. The method of claim 1, wherein the speech input includes an input sound indicative of the user-defined keyword.

9. The method of claim 8, wherein determining the sequence of subwords comprises generating the sequence of subwords from the input sound.

10. The method of claim 1, wherein the sequence of subwords includes at least one of phones, phonemes, triphones, and syllables.

11. The method of claim 1, further comprising:

generating at least one mixed sample sound by mixing the speech input with at least one type of noise.

12. The method of claim 11, wherein determining the sequence of subwords is further based on the at least one mixed sample sound.

13. The method of claim 11, wherein modifying the at least one subword model is based on the at least one mixed sample sound.

14. The method of claim 1, wherein generating the keyword model further comprises including a silence portion as a subword unit at a beginning or an end of the sequence of subwords.

15. The method of claim 3, wherein the function comprises a voice assistant application configured to output a message.

16. The method of claim 1, further comprising:

prompting the user to speak a particular sentence that is phonetically balanced;

receiving speech input indicative of the particular sentence; and

adjusting the at least one subword model based on the speech input indicative of the particular sentence.

17. The method of claim 1, wherein determining the sequence of subwords comprises deteiiiiining at least two sequences of subwords from the speech input and selecting one of the at least two sequences of subwords as the sequence of subwords.

18. An electronic device for generating a keyword model associated with a user-defined keyword, comprising:

an input unit configured to receive a speech input from a user, at a sensor of the electronic device, in response to a prompt;

a subword recognition unit configured to extract a speech segment from the speech input; and

a user-defined keyword model generation unit configured to;

determine a sequence of subwords representative of the speech segment of the speech input;

obtain at least one subword model of a plurality of subword models based on the sequence of subwords, wherein the at least one subword model includes a plurality of subword units and model parameters associated with each of the plurality of subword units;

generating the keyword model associated with the user-defined keyword by;

modify the plurality of subword units of the at least one subword model based on the sequence of subwords representative of the speech segment of the speech input; and

adjust the model parameters of the at least one subword model based on acoustic features extracted from the speech segment of the speech input; and

provide the keyword model associated with the user-defined keyword to a voice activation unit.

19. The electronic device of claim 18, wherein the input unit includes a sound sensor configured to receive at least one input indicative of the user-defined keyword.

20. The electronic device of claim 19, wherein the subword recognition unit is further configured to detect, based on the keyword model associated with the user-defined keyword, the user-defined keyword in the at least one input.

21. The electronic device of claim 18, further comprising a mixed sound generation unit configured to generate at least one mixed sample sound, the at least one mixed sound sample includes the speech input with at least one type of noise.

22. The electronic device of claim 18, wherein the voice activation unit is configured to detect the user-defined keyword in an input sound based on the keyword model associated with the user-defined keyword.

23. The electronic device of claim 18, wherein the voice activation unit is configured to perform a function associated with the user-defined keyword to activate an application associated with the user-defined keyword.

24. The electronic device of claim 18, further comprising a pre-adaptation unit configured to include a silence portion as a subword unit at a beginning or an end of the sequence of subwords.

25. The electronic device of claim 18, wherein the user-defined keyword model generation unit is configured to determine a threshold score of the keyword model associated with the user-defined keyword.

26. The electronic device of claim 25, wherein the user-defined keyword model generation unit is configured to:

determine a matching score of a test input sound indicative of the user-defined keyword based on the keyword model associated with the user-defined keyword; and

adjust the threshold score of the keyword model associated with the user-defined keyword based on the matching score.

27. A non-transitory computer-readable storage medium storing instructions for generating a keyword model associated with a user-defined keyword in an electronic device, the instructions causing a processor to perform operations of:

receiving a speech input from a user, at a sensor of the electronic device, in response to a prompt;

extracting a speech segment from the speech input;

determining a sequence of subwords representative of the speech segment of the speech input;

obtaining at least one subword model based on the sequence of subwords, wherein the at least one subword model includes a plurality of subword units and model parameters associated with each of the plurality of subword units;

generating the keyword model associated with the user-defined keyword by;

modifying the plurality of subword units of the at least one subword model based on the sequence of subwords representative of the speech segment of the speech input; and

adjusting the model parameters of the at least one subword model based on acoustic features extracted from the speech segment of the speech input; and

providing the keyword model associated with the user-defined keyword to a voice activation unit.

28. An electronic device for generating a keyword model associated with a user-defined keyword, comprising:

means for receiving a speech input from a user, at a sensor of the electronic device, in response to a prompt;

means for extracting a speech segment from the speech input;

means for determining a sequence of subwords representative of the speech segment of the speech input;

means for obtaining at least one subword model of a plurality of subword models based on the sequence of subwords, wherein the at least one subword model includes a plurality of subword units and model parameters associated with each of the plurality of subword units;

means for generating the keyword model associated with the user-defined keyword by;

modifying the plurality of subword units of the at least one subword model based on the sequence of subwords representative of the speech segment of the speech input; and

adjusting the model parameters of the at least one subword model based on acoustic features extracted from the speech segment of the speech input; and

means for providing the keyword model associated with the user-defined keyword to a voice activation unit.

29. The electronic device of claim 28, wherein the speech input includes at least one sample sound indicative of the user-defined keyword.

30. The electronic device of claim 28, further comprising:

means for generating at least one mixed sample sound by mixing the speech input with at least one type of noise.