USER-SPECIFIC ACOUSTIC MODELS

• US 20180330737A1
• Filed: 09/22/2017
• Published: 11/15/2018
• Est. Priority Date: 05/12/2017
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: PAULIK, Matthias, MASON, Henry G., SKINDER, Jason A.
• Granted Patent: US 10,410,637 B2
• Time in Patent Office: Days
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/07 to the speaker
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 17/04 Training, enrolment or model building
• CPC Class Codes: G10L 2015/0635 updating or merging of old and new templates; Mean values; Weighting
• CPC Class Codes: G10L 2015/0636 Threshold criteria for the updating

Abstract

Systems and processes for providing user-specific acoustic models are provided. In accordance with one example, a method includes, at an electronic device having one or more processors, receiving a plurality of speech inputs, each of the speech inputs associated with a same user of the electronic device; providing each of the plurality of speech inputs to a user-independent acoustic model, the user-independent acoustic model providing a plurality of speech results based on the plurality of speech inputs; initiating a user-specific acoustic model on the electronic device; and adjusting the user-specific acoustic model based on the plurality of speech inputs and the plurality of speech results.

Claims

1. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of a first electronic device, cause the first electronic device to:

receive a plurality of speech inputs, each of the speech inputs associated with a same user of the electronic device;

provide each of the plurality of speech inputs to a user-independent acoustic model, the user-independent acoustic model, the user-independent acoustic model providing a plurality of speech results based on the plurality of speech inputs;

initiate a user-specific acoustic model on the electronic device;

adjust the user-specific acoustic model based on the plurality of speech inputs and the plurality of speech results;

provide the user-specific acoustic model to another electronic device;

wherein at the another electronic device;

receive the user-specific acoustic model;

receive a speech input; and

identify, with the user-specific acoustic model, a speaker of the speech input.

3. The non-transitory computer-readable storage medium of claim 1, wherein providing the user-specific acoustic model to another electronic device comprises:

determining whether the user-specific acoustic model has been trained on a threshold number of speech inputs;

in accordance with a determination that the user-specific acoustic model has been trained on a threshold number of speech inputs, providing the user-specific acoustic model to the another electronic device; and

in accordance with a determination that the user-specific acoustic model has not been trained on a threshold number of speech inputs;

adjusting the user-specific model based on a second plurality of speech inputs and a second plurality of speech results; and

providing the user-specific acoustic model to the another electronic device.

5. The non-transitory computer-readable storage medium of claim 1, wherein identifying, with the user-specific acoustic model, a speaker of the speech input comprises:

providing the speech input to the user-specific acoustic model to provide a first speech result and a first accuracy score corresponding to the speech result;

providing the speech input to another user-specific acoustic model to provide a second speech result and a second accuracy score corresponding to the speech result; and

identifying the speaker of the speech input based on the first accuracy score and the second accuracy score.

6. The non-transitory computer-readable storage medium of claim 1, wherein receiving a plurality of speech inputs, each of the speech inputs associated with a same user of the electronic device comprises:

receiving one or more speech inputs of the plurality of speech inputs from the another electronic device.

7. The non-transitory computer-readable storage medium of claim 1, wherein receiving a plurality of speech inputs comprises:

receiving one or more speech inputs of the plurality of speech inputs at the electronic device.

8. The non-transitory computer-readable storage medium of claim 7, wherein receiving one or more speech inputs of the plurality of speech inputs at the electronic device comprises:

obtaining the one or more speech inputs of the plurality of speech inputs from a user utterance corresponding to a phone call.

9. The non-transitory computer-readable storage medium of claim 7, wherein receiving one or more speech inputs of the plurality of speech inputs at the electronic device comprises:

obtaining the one or more speech inputs of the plurality of speech inputs from a user utterance corresponding to a request for a digital assistant.

10. The non-transitory computer-readable storage medium of claim 1, wherein the user-independent acoustic model is based on a dataset, and

wherein initializing the user-specific acoustic model comprises:

initializing the user-specific acoustic model using the dataset.

11. The non-transitory computer-readable storage medium of claim 1, wherein the user-independent acoustic model has a first number of parameters and the user-specific acoustic model has a second number of parameters, wherein the first number is greater than the second number.

12. The non-transitory computer-readable storage medium of claim 1, wherein the user-independent acoustic model is an ensemble of two or more acoustic models.

2. (canceled)

4. (canceled)

13. A method, comprising:

at an electronic device having one or more processors;

receiving a plurality of speech inputs, each of the speech inputs associated with a same user of the electronic device;

providing each of the plurality of speech inputs to a user-independent acoustic model, the user-independent acoustic model providing a plurality of speech results based on the plurality of speech inputs;

initiating a user-specific acoustic model on the electronic device;

adjusting the user-specific acoustic model based on the plurality of speech inputs and the plurality of speech results;

providing the user-specific acoustic model to another electronic device;

wherein at the another electronic device;

receiving the user-specific acoustic model;

receiving a speech input; and

identifying, with the user-specific acoustic model, a speaker of the speech input.

15. The method of claim 13, wherein providing the user-specific acoustic model to another electronic device comprises:

determining whether the user-specific acoustic model has been trained on a threshold number of speech inputs;

in accordance with a determination that the user-specific acoustic model has been trained on a threshold number of speech inputs, providing the user-specific acoustic model to the another electronic device; and

in accordance with a determination that the user-specific acoustic model has not been trained on a threshold number of speech inputs;

adjusting the user-specific model based on a second plurality of speech inputs and a second plurality of speech results; and

providing the user-specific acoustic model to the another electronic device.

17. The method of claim 13, wherein identifying, with the user-specific acoustic model, a speaker of the speech input comprises:

providing the speech input to the user-specific acoustic model to provide a first speech result and a first accuracy score corresponding to the speech result;

providing the speech input to another user-specific acoustic model to provide a second speech result and a second accuracy score corresponding to the speech result; and

identifying the speaker of the speech input based on the first accuracy score and the second accuracy score.

18. The method of claim 13, wherein receiving a plurality of speech inputs, each of the speech inputs associated with a same user of the electronic device comprises:

receiving one or more speech inputs of the plurality of speech inputs from the another electronic device.

19. The method of claim 13, wherein receiving a plurality of speech inputs comprises:

receiving one or more speech inputs of the plurality of speech inputs at the electronic device.

20. The method of claim 19, wherein receiving one or more speech inputs of the plurality of speech inputs at the electronic device comprises:

obtaining the one or more speech inputs of the plurality of speech inputs from a user utterance corresponding to a phone call.

21. The method of claim 19, wherein receiving one or more speech inputs of the plurality of speech inputs at the electronic device comprises:

obtaining the one or more speech inputs of the plurality of speech inputs from a user utterance corresponding to a request for a digital assistant.

22. The method of claim 13, wherein the user-independent acoustic model is based on a dataset, and

wherein initializing the user-specific acoustic model comprises:

initializing the user-specific acoustic model using the dataset.

23. The method of claim 13, wherein the user-independent acoustic model has a first number of parameters and the user-specific acoustic model has a second number of parameters, wherein the first number is greater than the second number.

24. The method of claim 13, wherein the user-independent acoustic model is an ensemble of two or more acoustic models.

14. (canceled)

16. (canceled)

25. An electronic device, comprising:

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

receiving a plurality of speech inputs, each of the speech inputs associated with a same user of the electronic device;

providing each of the plurality of speech inputs to a user-independent acoustic model, the user-independent acoustic model providing a plurality of speech results based on the plurality of speech inputs;

initiating a user-specific acoustic model on the electronic device;

adjusting the user-specific acoustic model based on the plurality of speech inputs and the plurality of speech results;

providing the user-specific acoustic model to another electronic device;

wherein at the another electronic device;

receiving the user-specific acoustic model;

receiving a speech input; and

identifying, with the user-specific acoustic model, a speaker of the speech input.

27. The electronic device of claim 25, wherein providing the user-specific acoustic model to another electronic device comprises:

determining whether the user-specific acoustic model has been trained on a threshold number of speech inputs;

in accordance with a determination that the user-specific acoustic model has been trained on a threshold number of speech inputs, providing the user-specific acoustic model to the another electronic device; and

in accordance with a determination that the user-specific acoustic model has not been trained on a threshold number of speech inputs;

adjusting the user-specific model based on a second plurality of speech inputs and a second plurality of speech results; and

providing the user-specific acoustic model to the another electronic device.

29. The electronic device of claim 25, wherein identifying, with the user-specific acoustic model, a speaker of the speech input comprises:

providing the speech input to the user-specific acoustic model to provide a first speech result and a first accuracy score corresponding to the speech result;

providing the speech input to another user-specific acoustic model to provide a second speech result and a second accuracy score corresponding to the speech result; and

identifying the speaker of the speech input based on the first accuracy score and the second accuracy score.

30. The electronic device of claim 25, wherein receiving a plurality of speech inputs, each of the speech inputs associated with a same user of the electronic device comprises:

receiving one or more speech inputs of the plurality of speech inputs from the another electronic device.

31. The electronic device of claim 25, wherein receiving a plurality of speech inputs comprises:

receiving one or more speech inputs of the plurality of speech inputs at the electronic device.

32. The electronic device of claim 31, wherein receiving one or more speech inputs of the plurality of speech inputs at the electronic device comprises:

obtaining the one or more speech inputs of the plurality of speech inputs from a user utterance corresponding to a phone call.

33. The electronic device of claim 31, wherein receiving one or more speech inputs of the plurality of speech inputs at the electronic device comprises:

obtaining the one or more speech inputs of the plurality of speech inputs from a user utterance corresponding to a request for a digital assistant.

34. The electronic device of claim 25, wherein the user-independent acoustic model is based on a dataset, and

wherein initializing the user-specific acoustic model comprises:

initializing the user-specific acoustic model using the dataset.

35. The electronic device of claim 25, wherein the user-independent acoustic model has a first number of parameters and the user-specific acoustic model has a second number of parameters, wherein the first number is greater than the second number.

36. The electronic device of claim 25, wherein the user-independent acoustic model is an ensemble of two or more acoustic models.

26. (canceled)

28. (canceled)