CACHING APPARATUS FOR SERVING PHONETIC PRONUNCIATIONS

• US 20160093298A1
• Filed: 08/25/2015
• Published: 03/31/2016
• Est. Priority Date: 09/30/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: NAIK, Devang K., MOHAMED, Ali S., CHEN, Hong M.
• Granted Patent: US 9,646,609 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/005 Language recognition
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/228 of application context

Abstract

Systems and processes for generating a shared pronunciation lexicon and using the shared pronunciation lexicon to interpret spoken user inputs received by a virtual assistant are provided. In one example, the process can include receiving pronunciations for words or named entities from multiple users. The pronunciations can be tagged with context tags and stored in the shared pronunciation lexicon. The shared pronunciation lexicon can then be used to interpret a spoken user input received by a user device by determining a relevant subset of the shared pronunciation lexicon based on contextual information associated with the user device and performing speech-to-text conversion on the spoken user input using the determined subset of the shared pronunciation lexicon.

Claims

1. A method for operating a virtual assistant, the method comprising:

receiving, from a first user device, a pronunciation for a first named entity;

storing the pronunciation for the first named entity in a shared pronunciation lexicon;

receiving, from a second user device, a pronunciation for a second named entity; and

storing the pronunciation for the second named entity in the shared pronunciation lexicon.

2. The method of claim 1, wherein storing the pronunciation for the first named entity in the shared pronunciation lexicon comprises:

determining one or more context tags for the pronunciation for the first named entity; and

storing the pronunciation for the first named entity in association with the determined one or more context tags.

3. The method of claim 2, wherein the one or more context tags comprise a location tag identifying a location associated with the pronunciation for the first named entity.

4. The method of claim 2, wherein the one or more context tags comprise a domain tag identifying a subject matter domain associated with the pronunciation for the first named entity.

5. The method of claim 2, wherein the one or more context tags comprise a language tag identifying a language associated with the pronunciation for the first named entity.

6. The method of claim 1, wherein the method further comprises:

receiving, from a third user device, a second pronunciation for the first named entity; and

storing the second pronunciation for the first named entity in the shared pronunciation lexicon.

7. The method of claim 1, wherein the method further comprises:

receiving, from a fourth user device, a context of the fourth user device;

determining a relevant subset of the shared pronunciation lexicon based on the context of the fourth user device;

receiving, from the fourth user device, audio data representing user speech; and

performing speech-to-text conversion on the audio data using the determined relevant subset of the shared pronunciation lexicon to generate a textual representation of the user speech.

8. The method of claim 7, wherein performing speech-to-text conversion on the audio data using the determined relevant subset of the shared pronunciation lexicon excludes the use of portions of the shared pronunciation lexicon not included in the determined relevant subset of the shared pronunciation lexicon.

9. The method of claim 7, wherein determining the relevant subset of the shared pronunciation lexicon based on the context of the fourth user device comprises determining to include, in the relevant subset of the shared pronunciation lexicon, one or more pronunciations for named entities in the shared pronunciation lexicon that are associated with a context tag related to the context of the fourth user device.

10. The method of claim 7, wherein the context of the fourth user device comprises a contact list stored on the fourth user device, and wherein determining the relevant subset of the shared pronunciation lexicon based on the context of the fourth user device comprises determining to include, in the relevant subset of the shared pronunciation lexicon, one or more pronunciations for named entities in the contact list that are also in the shared pronunciation lexicon.

11. The method of claim 7, further comprising:

determining a response to the user speech, wherein the response comprises a named entity;

determining a pronunciation for the named entity using the determined relevant subset of the shared pronunciation lexicon; and

transmitting the response and the pronunciation for the named entity to the fourth user device.

12. The method of claim 1, further comprising:

deleting one or more pronunciations for a named entity in the shared pronunciation lexicon that has least recently been accessed.

13. The method of claim 1, wherein the pronunciation for the first named entity comprises an audio recording of a user associated with the first user device speaking the first named entity, and wherein storing the pronunciation for the first named entity in the shared pronunciation lexicon comprises:

generating an acoustic model representing the pronunciation for the first named entity based on the audio recording, and

storing the acoustic model in the shared pronunciation lexicon.

14. The method of claim 1, wherein the pronunciation for the first named entity comprises an acoustic model representing the pronunciation for the first named entity generated by the first user device.

15. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, when executed by one or more processors, cause the one or more processors to:

receive, from a first user device, a pronunciation for a first named entity;

store the pronunciation for the first named entity in a shared pronunciation lexicon;

receive, from a second user device, a pronunciation for a second named entity; and

store the pronunciation for the second named entity in the shared pronunciation lexicon.

16. A system comprising:

one or more processors;

memory storing one or more programs, the one or more programs comprising instructions, when executed by the one or more processors, cause the one or more processors to;

receive, from a first user device, a pronunciation for a first named entity;

store the pronunciation for the first named entity in a shared pronunciation lexicon;

receive, from a second user device, a pronunciation for a second named entity; and

store the pronunciation for the second named entity in the shared pronunciation lexicon.

17. The system of claim 16, wherein storing the pronunciation for the first named entity in the shared pronunciation lexicon comprises:

determining one or more context tags for the pronunciation for the first named entity; and

storing the pronunciation for the first named entity in association with the determined one or more context tags.

18. The system of claim 16, wherein the instructions further cause the one or more processors to:

receive, from a third user device, a second pronunciation for the first named entity; and

store the second pronunciation for the first named entity in the shared pronunciation lexicon.

19. The system of claim 16, wherein the instructions further cause the one or more processors to:

receive, from a fourth user device, a context of the fourth user device;

determine a relevant subset of the shared pronunciation lexicon based on the context of the fourth user device;

receive, from the fourth user device, audio data representing user speech; and

perform speech-to-text conversion on the audio data using the determined relevant subset of the shared pronunciation lexicon to generate a textual representation of the user speech.

20. The system of claim 19, wherein performing speech-to-text conversion on the audio data using the determined relevant subset of the shared pronunciation lexicon excludes the use of portions of the shared pronunciation lexicon not included in the determined relevant subset of the shared pronunciation lexicon.

21. The system of claim 19, wherein determining the relevant subset of the shared pronunciation lexicon based on the context of the fourth user device comprises determining to include, in the relevant subset of the shared pronunciation lexicon, one or more pronunciations for named entities in the shared pronunciation lexicon that are associated with a context tag related to the context of the fourth user device.

22. The system of claim 19, wherein the context of the fourth user device comprises a contact list stored on the fourth user device, and wherein determining the relevant subset of the shared pronunciation lexicon based on the context of the fourth user device comprises determining to include, in the relevant subset of the shared pronunciation lexicon, one or more pronunciations for named entities in the contact list that are also in the shared pronunciation lexicon.

23. The system of claim 19, wherein the instructions further cause the one or more processors to:

determine a response to the user speech, wherein the response comprises a named entity;

determine a pronunciation for the named entity using the determined relevant subset of the shared pronunciation lexicon; and

transmit the response and the pronunciation for the named entity to the fourth user device.

24. The system of claim 16, wherein the instructions further cause the one or more processors to:

delete one or more pronunciations for a named entity in the shared pronunciation lexicon that has least recently been accessed.

25. The system of claim 16, wherein the pronunciation for the first named entity comprises an audio recording of a user associated with the first user device speaking the first named entity, and wherein storing the pronunciation for the first named entity in the shared pronunciation lexicon comprises:

generating an acoustic model representing the pronunciation for the first named entity based on the audio recording, and

storing the acoustic model in the shared pronunciation lexicon.