USER TRAINING BY INTELLIGENT DIGITAL ASSISTANT

• US 20140272821A1
• Filed: 03/14/2014
• Published: 09/18/2014
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: PITSCHEL, Donald W., GRUBER, Thomas R.
• Granted Patent: US 11,151,899 B2
• Time in Patent Office: Days
• US Class Current: 434/157
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G09B 19/06 Foreign languages with audible presentation of material to be studied G09B5/04
• CPC Class Codes: G10L 2015/225 Feedback of the input speech

Abstract

The method includes receiving, from a user, a first speech input spoken in a first language; inferring a user intent based on at least the first speech input in the first language; based on the inferred user intent, generating one or more alternative expressions of the first speech input in the first language; and providing feedback to the user introducing the alternative expressions as a more preferred input to express the inferred user intent than the first speech input provided by the user.

Claims

1. A computer-implemented method for operating a digital assistant, comprising:

at a device having one or more processors and memory;

receiving, from a user, a first speech input spoken in a first language;

inferring a user intent based on at least the first speech input in the first language;

based on the inferred user intent, generating one or more alternative expressions of the first speech input in the first language; and

providing feedback to the user introducing the alternative expressions as a more preferred input to express the inferred user intent than the first speech input provided by the user.

2. The method of claim 1, wherein providing the feedback further comprises:

providing the feedback in a second language different from the first language, wherein the second language is a primary language associated with the user, and the first language is a secondary language associated with the user.

3. The method of claim 1, wherein the one or more alternative expressions of the first speech input includes at least a respective alternative expression that corrects a pronunciation of at least one word in the first speech input.

4. The method of claim 1, wherein the one or more alternative expressions of the first speech input includes at least a respective alternative expression that corrects a grammatical usage of at least one word in the first speech input.

5. The method of claim 1, wherein the one or more alternative expressions of the first speech input includes at least a respective alternative expression that replaces at least one word or phrase in the first speech input with another word or phrase.

6. The method of claim 1, wherein inferring the user intent based on the first speech input in the first language further comprises:

identifying a customized speech-to-text model of the first language for the user, wherein the customized speech-to-text model has been established based on training samples provided by native speakers of a second language of which the user is also a native speaker;

processing the first speech input to generate a text string using the customized speech-to-text model; and

using the text string as input for an intent inference model of the digital assistant.

7. The method of claim 1, wherein generating the one or more alternative expressions of the first speech input in the first language further comprises:

identifying a second speech input previously provided by a native speaker of the first language, wherein the second speech input had been associated with a respective user intent that is identical to the inferred user intent of the first speech input, and wherein a task flow executed for the respective user intent had been satisfactory to said native speaker; and

utilizing the second speech input as one of the alternative expressions of the first speech input.

8. The method of claim 1, wherein providing the feedback to the user introducing the alternative expressions as a more preferred input to express the inferred user intent further comprises:

providing, in a second language, an explanation of a difference between a first alternative expression and the first speech input, wherein the second language is a primary language associated with the user, and the first language is a secondary language associated with the user.

9. The method of claim 1, further comprising:

receiving a second speech input in the first language from the user, the second speech input utilizing at least one of the alternative expressions;

determining whether the second speech input is a satisfactory vocal utterance of the at least one alternative expression; and

upon determining that the second speech input is a satisfactory vocal utterance of the at least one alternative expression, executing a task flow to fulfill the inferred user intent.

10. The method of claim 1, further comprising:

providing, in a second language, a paraphrase of the first speech input based on the inferred user intent to confirm the correctness of the inferred user intent, wherein the digital assistant generates the alternative expressions and provides the feedback after receiving user confirmation that the inferred user intent is the correct user intent.

11. The method of claim 1, wherein inferring the user intent based on at least the first speech input in the first language further comprises:

inferring the user intent further based on a current context associated with the user.

12. A computer-implemented method for operating a digital assistant, comprising:

at a device having one or more processors and memory;

receiving, from a user, a first speech input spoken in a first language;

inferring a user intent based on at least the first speech input in the first language;

based on the inferred user intent, generating one or more alternative expressions of the first speech input in the first language; and

providing feedback to the user introducing the alternative expressions as a more preferred input to express the inferred user intent than the first speech input provided by the user; and

providing at least a command mode and a foreign language training mode, wherein the digital assistant executes a task flow to fulfill the inferred user intent in the command mode, and wherein the digital assistant generates the one or more alternative expressions and provides the feedback to the user in the foreign language training mode.

13. The method of claim 12, further comprising:

concurrently providing both the command mode and the foreign language training mode, wherein the digital assistant executes the task flow to fulfill the inferred user intent, in addition to generating the one or more alternative expressions and providing the feedback to the user.

14. The method of claim 12, further comprising:

receiving user selection of the foreign language training mode; and

enabling the foreign language training mode in response to the user selection of the foreign language training mode.

15. The method of claim 12, further comprising:

automatically, without user intervention, enabling the foreign language training mode based on a current location of the user, wherein a primary language associated with the current location of the user is the first language.

16. A non-transitory computer-readable medium having instructions stored thereon, the instructions, when executed by one or more processors, cause the processors to perform operations comprising:

receiving, from a user, a first speech input spoken in a first language;

inferring a user intent based on at least the first speech input in the first language;

based on the inferred user intent, generating one or more alternative expressions of the first speech input in the first language; and

providing feedback to the user introducing the alternative expressions as a more preferred input to express the inferred user intent than the first speech input provided by the user.

17. The non-transitory computer-readable medium of claim 16, wherein providing the feedback further comprises:

providing the feedback in a second language different from the first language, wherein the second language is a primary language associated with the user, and the first language is a secondary language associated with the user.

18. The non-transitory computer-readable medium of claim 16, wherein the one or more alternative expressions of the first speech input includes at least a respective alternative expression that corrects a pronunciation of at least one word in the first speech input.

19. The non-transitory computer-readable medium of claim 16, wherein the one or more alternative expressions of the first speech input includes at least a respective alternative expression that corrects a grammatical usage of at least one word in the first speech input.

20. The non-transitory computer-readable medium of claim 16, wherein the one or more alternative expressions of the first speech input includes at least a respective alternative expression that replaces at least one word or phrase in the first speech input with another word or phrase.

21. The non-transitory computer-readable medium of claim 16, further comprising instructions for providing at least a command mode and a foreign language training mode, wherein the digital assistant executes a task flow to fulfill the inferred user intent in the command mode, and wherein the digital assistant generates the one or more alternative expressions and provides the feedback to the user in the foreign language training mode.

22. A system, comprising

one or more processors;

and

memory having instructions stored thereon, the instructions, when executed by one or more processors, cause the processors to perform operations comprising;

receiving, from a user, a first speech input spoken in a first language;

inferring a user intent based on at least the first speech input in the first language;

based on the inferred user intent, generating one or more alternative expressions of the first speech input in the first language; and

providing feedback to the user introducing the alternative expressions as a more preferred input to express the inferred user intent than the first speech input provided by the user.

23. The system of claim 22, wherein providing the feedback further comprises:

providing the feedback in a second language different from the first language, wherein the second language is a primary language associated with the user, and the first language is a secondary language associated with the user.

24. The system of claim 22, wherein the one or more alternative expressions of the first speech input includes at least a respective alternative expression that corrects a pronunciation of at least one word in the first speech input.

25. The system of claim 22, wherein the one or more alternative expressions of the first speech input includes at least a respective alternative expression that corrects a grammatical usage of at least one word in the first speech input.

26. The system of claim 22, wherein the one or more alternative expressions of the first speech input includes at least a respective alternative expression that replaces at least one word or phrase in the first speech input with another word or phrase.

27. The system of claim 22, further comprising instructions for providing at least a command mode and a foreign language training mode, wherein the digital assistant executes a task flow to fulfill the inferred user intent in the command mode, and wherein the digital assistant generates the one or more alternative expressions and provides the feedback to the user in the foreign language training mode.