Automatic accent detection using acoustic models

• US 10,255,907 B2
• Filed: 09/04/2015
• Issued: 04/09/2019
• Est. Priority Date: 06/07/2015
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Nallasamy, Udhyakumar, Kajarekar, Sachin S., Paulik, Matthias, Seigel, Matthew
• Primary Examiner(s): Opsasnick, Michael N
• Application Number:
  US14/846,650
• Publication Number:
  US 20160358600A1
• Time in Patent Office: 1,313 Days
• Field of Search: 704231, 704243
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/07 to the speaker
• CPC Class Codes: G10L 15/10 using distance or distortion measures between unknown speech and reference templates
• CPC Class Codes: G10L 15/16 using artificial neural networks
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

Systems and processes for automatic accent detection are provided. In accordance with one example, a method includes, at an electronic device with one or more processors and memory, receiving a user input, determining a first similarity between a representation of the user input and a first acoustic model of a plurality of acoustic models, and determining a second similarity between the representation of the user input and a second acoustic model of the plurality of acoustic models. The method further includes determining whether the first similarity is greater than the second similarity. In accordance with a determination that the first similarity is greater than the second similarity, the first acoustic model may be selected; and in accordance with a determination that the first similarity is not greater than the second similarity, the second acoustic model may be selected.

Claims

1. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to:

receive a user input;

determine a first similarity between a representation of the user input and a first acoustic model of a plurality of acoustic models, wherein the first acoustic model is associated with a first accent of a language;

determine a second similarity between the representation of the user input and a second acoustic model of the plurality of acoustic models, wherein the second acoustic model is associated with a second accent of the language;

determine whether the first similarity is greater than the second similarity;

in accordance with a determination that the first similarity is greater than the second similarity, select the first acoustic model;

in accordance with a determination that the first similarity is not greater than the second similarity, select the second acoustic model;

associate the selected acoustic model with a user profile corresponding to a speaker;

receive a second user input from the speaker; and

determine a representation of the second user input using the selected acoustic model associated with the user profile corresponding to the speaker.

2. The non-transitory computer-readable storage medium of claim 1, wherein the user input comprises a plurality of user utterances.

3. The non-transitory computer-readable storage medium of claim 1, wherein determining whether the first similarity is greater than the second similarity includes determining whether a distance between the representation of the user input and the first acoustic model is greater than a distance between the representation of the user input and the second acoustic model.

4. The non-transitory computer-readable storage medium of claim 3, wherein determining whether a distance between the representation of the user input and the first acoustic model is greater than a distance between the representation of the user input and the second acoustic model includes:

determining a first angle between the representation of the user input and the first acoustic model; and

determining a second angle between the representation of the user input and the second acoustic model,

wherein the distance between the representation of the user input and the first acoustic model is based on the first angle and wherein the distance between the representation of the user input and the second acoustic model is based on the second angle.

5. The non-transitory computer-readable storage medium of claim 1, wherein the first acoustic model of the plurality of acoustic models is associated with a first level of a data structure and the second acoustic model of the plurality of acoustic models is associated with a second level of the data structure.

6. The non-transitory computer-readable storage medium of claim 5, wherein the first acoustic model is a first neural network acoustic model and the second acoustic model is a second neural network acoustic model and wherein the first and second neural network acoustic models share a same neural network layer.

7. The non-transitory computer-readable storage medium of claim 5, wherein the second acoustic model is adapted from the first acoustic model using MLLR adaption, cMLLR adaption, MAP adaption, Eigenvoice adaption, or a combination thereof.

21. The non-transitory computer-readable storage medium of claim 1, wherein the one or more programs further comprise instructions, which when executed by one or more processors of the electronic device, cause the electronic device to:

in response to selecting the first acoustic model;

generate a first recognition result based on at least the user input and the first acoustic model; and

output the first recognition result;

in response to selecting the second acoustic model;

generate a second recognition result based on at least the user input and the second acoustic model; and

output the second recognition result.

24. The non-transitory computer-readable storage medium of claim 1, wherein the representation of the user input is a vector representation.

8. A method, comprising:

at an electronic device with one or more processors and memory;

receiving a user input;

determining a first similarity between a representation of the user input and a first acoustic model of a plurality of acoustic models, wherein the first acoustic model is associated with a first accent of a language;

determining a second similarity between the representation of the user input and a second acoustic model of the plurality of acoustic models, wherein the second acoustic model is associated with a second accent of the language;

determining whether the first similarity is greater than the second similarity;

in accordance with a determination that the first similarity is greater than the second similarity, selecting the first acoustic model;

in accordance with a determination that the first similarity is not greater than the second similarity, selecting the second acoustic model;

associating the selected acoustic model with a user profile corresponding to a speaker;

receiving a second user input from the speaker; and

determining a representation of the second user input using the selected acoustic model associated with the user profile corresponding to the speaker.

9. The method of claim 8, wherein the user input comprises a plurality of user utterances.

10. The method of claim 8, wherein determining whether the first similarity is greater than the second similarity includes determining whether a distance between the representation of the user input and the first acoustic model is greater than a distance between the representation of the user input and the second acoustic model.

11. The method of claim 10, wherein determining whether a distance between the representation of the user input and the first acoustic model is greater than a distance between the representation of the user input and the second acoustic model includes:

determining a first angle between the representation of the user input and the first acoustic model; and

determining a second angle between the representation of the user input and the second acoustic model,

wherein the distance between the representation of the user input and the first acoustic model is based on the first angle and wherein the distance between the representation of the user input and the second acoustic model is based on the second angle.

12. The method of claim 8, wherein the first acoustic model of the plurality of acoustic models is associated with a first level of a data structure and the second acoustic model of the plurality of acoustic models is associated with a second level of the data structure.

13. The method of claim 12, wherein the first acoustic model is a first neural network acoustic model and the second acoustic model is a second neural network acoustic model and wherein the first and second neural network acoustic models share a same neural network layer.

14. The method of claim 12, wherein the second acoustic model is adapted from the first acoustic model using MLLR adaption, cMLLR adaption, MAP adaption, Eigenvoice adaption, or a combination thereof.

22. The method of claim 8, further comprising:

in response to selecting the first acoustic model;

generating a first recognition result based on at least the user input and the first acoustic model; and

outputting the first recognition result;

in response to selecting the second acoustic model;

generating a second recognition result based on at least the user input and the second acoustic model; and

outputting the second recognition result.

15. An electronic device, comprising:

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

receiving a user input;

determining a first similarity between a representation of the user input and a first acoustic model of a plurality of acoustic models, wherein the first acoustic model is associated with a first accent of a language;

determining a second similarity between the representation of the user input and a second acoustic model of the plurality of acoustic models, wherein the second acoustic model is associated with a second accent of the language;

determining whether the first similarity is greater than the second similarity;

in accordance with a determination that the first similarity is greater than the second similarity, selecting the first acoustic model;

in accordance with a determination that the first similarity is not greater than the second similarity, selecting the second acoustic model;

associating the selected acoustic model with a user profile corresponding to a speaker;

receiving a second user input from the speaker; and

determining a representation of the second user input using the selected acoustic model associated with the user profile corresponding to the speaker.

16. The electronic device of claim 15, wherein the user input comprises a plurality of user utterances.

17. The electronic device of claim 15, wherein determining whether the first similarity is greater than the second similarity includes determining whether a distance between the representation of the user input and the first acoustic model is greater than a distance between the representation of the user input and the second acoustic model.

18. The electronic device of claim 17, wherein determining whether a distance between the representation of the user input and the first acoustic model is greater than a distance between the representation of the user input and the second acoustic model includes:

determining a first angle between the representation of the user input and the first acoustic model; and

determining a second angle between the representation of the user input and the second acoustic model,

wherein the distance between the representation of the user input and the first acoustic model is based on the first angle and wherein the distance between the representation of the user input and the second acoustic model is based on the second angle.

19. The electronic device of claim 15, wherein the first acoustic model of the plurality of acoustic models is associated with a first level of a data structure and the second acoustic model of the plurality of acoustic models is associated with a second level of the data structure.

20. The electronic device of claim 19, wherein the first acoustic model is a first neural network acoustic model and the second acoustic model is a second neural network acoustic model and wherein the first and second neural network acoustic models share a same neural network layer.

23. The electronic device of claim 15, wherein the one or more programs further include instructions for:

in response to selecting the first acoustic model;

generating a first recognition result based on at least the user input and the first acoustic model; and

outputting the first recognition result;

in response to selecting the second acoustic model;

generating a second recognition result based on at least the user input and the second acoustic model; and

outputting the second recognition result.