METHODS AND SYSTEMS FOR PHONETIC MATCHING IN DIGITAL ASSISTANT SERVICES

• US 20180336197A1
• Filed: 09/13/2017
• Published: 11/22/2018
• Est. Priority Date: 05/16/2017
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Skilling, Adrian, Hunt, Melvyn J., Evermann, Gunnar
• Granted Patent: US 10,403,278 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/10 using distance or distortion measures between unknown speech and reference templates
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units
• CPC Class Codes: G10L 2015/088 Word spotting

Abstract

Systems and processes for operating an intelligent automated assistant to provide media items based on phonetic matching techniques are provided. An example method includes receiving a speech input from a user and determining whether the speech input includes a user request for a media item. The method further includes, in accordance with a determination that the speech input includes a user request for obtaining a media item, determining a candidate media item from a plurality of media items. The method further includes determining, based on a difference between a phonetic representation of the candidate media item and a phonetic representation of the speech input, whether the candidate media item is to be provided to the user. The method further includes, in accordance with a determination that the candidate media item is to be provided to the user, providing the candidate media item to the user.

Claims

1. An electronic device comprising:

one or more processors;

memory; and

one or more programs stored in memory, the one or more programs including instructions for;

receiving a speech input from a user;

determining whether the speech input includes a user request for a media item;

in accordance with a determination that the speech input includes a user request for a media item, determining a candidate media item from a repository of media items;

determining, based on a difference between a phonetic representation of the candidate media item and a phonetic representation of the speech input, whether the candidate media item is to be provided to the user; and

in accordance with a determination that the candidate media item is to be provided to the user, providing the candidate media item to the user.

2. The electronic device of claim 1, wherein the speech input comprises unstructured natural language information.

3. The electronic device of claim 1, wherein determining whether the speech input includes a user request for a media item comprises:

obtaining a sequence of words based on the speech input; and

determining whether the sequence of words includes a representation of the user request for a media item.

4. The electronic device of claim 3, wherein determining whether the sequence of words includes a representation of the user request for a media item comprises one or more of:

determining whether the sequence of words includes one or more predetermined words; and

determining whether the sequence of words corresponds to one or more predetermined grammars.

5. The electronic device of claim 3, wherein determining whether the sequence of words includes a representation of the user request for a media item comprises determining whether the sequence of words includes one or more stop words.

6. The electronic device of claim 1, wherein determining the candidate media item from a repository of media items comprises:

generating a phonetic symbol sequence representing the speech input;

determining, based on the phonetic symbol sequence representing the speech input, one or more reference phonetic symbol sequences; and

determining the candidate media item based on the one or more determined reference phonetic symbol sequences.

7. The electronic device of claim 6, wherein generating a phonetic symbol sequence representing the speech input comprises:

generating the phonetic symbol sequence representing the speech input based on a sequence of words obtained from a speech-to-text conversion of the speech input.

8. The electronic device of claim 6, wherein the reference phonetic symbol sequences represent sequences of words obtained from one or more dictionaries.

9. The electronic device of claim 6, wherein determining, based on the phonetic symbol sequence representing the speech input, the one or more reference phonetic symbol sequences comprises:

determining, for each phonetic symbol of the phonetic symbol sequence representing the speech input, whether the phonetic symbol of the phonetic symbol sequence representing the speech input matches with a same reference phonetic symbol.

10. The electronic device of claim 6, wherein determining, based on the phonetic symbol sequence representing the speech input, the one or more reference phonetic symbol sequences comprises:

determining, for each phonetic symbol of the phonetic symbol sequence representing the speech input, whether the phonetic symbol of the phonetic symbol sequence representing the speech input corresponds to a different reference phonetic symbol.

11. The electronic device of claim 10, wherein determining whether the phonetic symbol of the phonetic symbol sequence representing the speech input corresponds to a different reference phonetic symbol comprises:

determining whether one or more predetermined operations are performed during the generation of a phonetic symbol sequence representing the speech input.

12. The electronic device of claim 11, wherein the one or more predetermined operations include one or more of:

a substitution operation, a deletion operation, and an insertion operation.

13. The electronic device of claim 6, further comprising:

determining, for each phonetic symbol of the phonetic symbol sequence representing the speech input, a probability that the phonetic symbol of the phonetic symbol sequence representing the speech input corresponds to a reference phonetic symbol.

14. The electronic device of claim 6, wherein determining the candidate media item based on the one or more determined reference phonetic symbol sequences comprises:

determining, for each of the one or more reference phonetic symbol sequences, a score indicating the degree of matching between the phonetic symbol sequence representing the speech input and each of the one or more reference phonetic symbol sequences; and

determining the candidate media item based on the scores associated with the one or more reference phonetic symbol sequences.

15. The electronic device of claim 14, wherein determining the candidate media item based on the scores associated with the one or more reference phonetic symbol sequences comprises:

ranking the one or more reference phonetic symbol sequences based on their respective scores;

selecting the reference phonetic symbol sequence having the highest score; and

identifying the candidate media item based on the reference phonetic symbol sequence having the highest score.

16. The electronic device of claim 15, further comprising:

determining whether the highest score satisfies a score-threshold condition;

in accordance with a determination that the highest score satisfies the score-threshold condition, foregoing the determination of whether the candidate media item is to be provided to the user.

17. The electronic device of claim 15, wherein identifying the candidate media item based on the reference phonetic symbol sequence having the highest score comprises:

mapping the reference phonetic symbol sequence having the highest score to phonetic representations of media items available in the repository of media items;

identifying the candidate media item based on the mapping.

18. The electronic device of claim 1, wherein determining whether the candidate media item is to be provided to the user comprises:

determining the difference between the phonetic representation of the candidate media item and the phonetic representation of the speech input;

determining whether the difference between the phonetic representation of the candidate media item and the phonetic representation of the speech input satisfies a threshold condition; and

in accordance with a determination that the difference satisfies the threshold condition, determining that the candidate media item is to be provided to the user.

19. The electronic device of claim 18, wherein determining the difference between the phonetic representation of the candidate media item and the phonetic representation of the speech input comprises:

determining the difference based on an expected score associated with a phonetic symbol sequence representing the speech input and a score indicating the degree of matching between the phonetic symbol sequence representing the speech input and a phonetic symbol sequence of the candidate media item.

20. The electronic device of claim 19, further comprising:

while determining whether the candidate media item is to be provided to the user, terminating the determination based on the expected score associated with the phonetic symbol sequence of the speech input.

21. The electronic device of claim 1, wherein providing the candidate media item to the user comprises outputting an audio associated with the candidate media item.

22. The electronic device of claim 1, further comprising:

determining additional information associated with the candidate media item; and

providing the additional information associated with the candidate media item to the user.

23. The electronic device of claim 1, prior to determining whether the speech input includes a user request for a media item, further comprising:

performing natural language processing of the speech input;

determining whether phonetic matching is required based on the results of the natural language processing;

in accordance with a determination that phonetic matching is required, initiating the determining of whether the speech input includes a user request for a media item; and

in accordance with a determination that phonetic matching is not required, providing a media item obtained based on the natural language processing results.

24. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to:

receive a speech input from a user;

determine whether the speech input includes a user request for a media item;

in accordance with a determination that the speech input includes a user request for obtaining a media item, determine a candidate media item from a repository of media items;

determine, based on a difference between a phonetic representation of the candidate media item and a phonetic representation of the speech input, whether the candidate media item is to be provided to the user; and

in accordance with a determination that the candidate media item is to be provided to the user, provide the candidate media item to the user.

25. A method for providing digital assistant service, comprising:

at one or more electronic devices with one or more processors and memory;

receiving a speech input from a user;

determining whether the speech input includes a user request for a media item;

in accordance with a determination that the speech input includes a user request for obtaining a media item, determining a candidate media item from a repository of media items;

determining, based on a difference between a phonetic representation of the candidate media item and a phonetic representation of the speech input, whether the candidate media item is to be provided to the user; and

in accordance with a determination that the candidate media item is to be provided to the user, providing the candidate media item to the user.